hross: May 2007 Archives

Based on feedback I've received from readers (Gerald Kanapathy and abenzvi -- thanks guys!) I've made some modifications to this post.

At some point while troubleshooting a problem, every ALUI portal administrator or developer is bound to ask themselves, "What's in a gateway URL?" and, more importantly, "How can I build one?"

Gateway URL’s look like the following:
http(s)://[portal_base]/gateway/PTARGS_spaceID_userID_objectID_communityID_pageID_classID/[gatewayed_URL]

Where portal_base is the base portal URL and gatewayed_URL is the URL you are trying to reach, which has been transformed using a function in the server API which I'll provide a reasonable copy of later.

A gatewayed URL, in many cases, only needs two pieces of information.:

• Class ID of calling object
• Object ID of calling object

The calling object is either a card, portlet or web service. Class IDs can be found in the java docs, among other places, but here they are:

• Card - 18
• Portlet - 43
• Service - 47

Leaving out community and user IDs could impact user preferences being passed to the back end, as Gerald pointed out. Object IDs can be found by browsing in the portal administratively (or using the EDK, etc). If you have access to the server API and/or UI source, methods can be found to construct gateway URLs under:

com.plumtree.portaluiinfrastructure.statichelpers.GatewayHelpers

The documentation for this class proves rather insightful (I guess that is no surprise), spelling out the rest of the arguments. I have provided it below, in case you don't have a copy handy (also see abenzvi's comment):

---

This class provides a set of static helpers for constructing valid gateway URLs. Gateway URLs follow a well-defined format to communicate information about the request to the gateway control. This section describes the format of the gateway URLs. For simplicity portal url is assumed to be http://machinename/portal/server.pt

The gateway URLs are identified by a special substring "gateway/PTARGS_" that should follow the portal URL. So, a general gateway URL looks like:

   http://machinename/portal/server.pt/gateway/PTARGS_1_2_3_4_5_6/http://remotemachine/url_to_gateway/somefile.html

where 1,2,3,4,5,6 are numeric arguments described below.

1. The first argument in an integer that contains a bit mask of gateway flags and type of associated gadget content that is being requested.
   • POPUPFLAG - notifies the gateway that requested page will be displayed in the popup window and therefore gateway should set the RETURN-URI to "close window" javascript.
   • RETURNFLAG - notifies the gateway that return URI is requested. The gateway will check that POPUPFLAG and either redirect to the last visited community or mypage or close the window.
   • REDIRECTGUEST - notifies the gateway that it should redirect guest users to the login page before displaying the request resource. (This is mostly used by Collaboration server)
2. The second argument is a numeric id of the user requesting the page. This argument is not used by the gateway and is intended for PPC PTTracker tool that analyzes the logs from the webserver. If user id is not known you can set it to any value, but 0 is preferred.
3. The third argument is a numeric id of the object that is being requested. For example, if you are requesting a page through gadget 201, then this argument should be set to 201.
4. The fourth argument is a community id.
5. The fifth argument is a page id.
6. The sixth argument is a resource type id. It defines what kind of object is identified by the object id provided in the third argument. Refer to Plumtree.idl to see what are numeric values for various resource types. Gateway supports the following resource types: Gadget, WebService, Card, DataSource.

---

You can use the somewhat incorrect code I have below to build gateway URLs (the CreateGatewayFriendlyURL function is correct, CreateGatewayUrl makes some assumptions), or you can take a look at some of the tags in the tag framework):

package com.plumtree.poc;

import java.net.URL;

public class GatewayHelper {
	public static String CreateGatewayFriendlyURL(String url) {
	        int position = url.indexOf('?');
	        String result = "";

	        if(-1 == position) {
	            result = url.replaceAll(";", "%3B");
	            result = result.replaceAll("\\%", "%25");
	            result = result.replaceAll("://", ";/");
	            result = result.replaceAll(":", ";");
	            return result;
	        } else {
	            String urlPath = url.substring(0, position);
	            result = urlPath.replaceAll(";", "%3B");
	            result = result.replaceAll("\\%", "%25");
	            result = result.replaceAll("://", ";/");
	            result = result.replaceAll(":", ";");
	            return result + url.substring(position);
	        }
	}
	
	public static String CreateGatewayUrl(URL currentUrl, int portletId, String docUrl) {
		String gatewayBase = currentUrl.getProtocol() + "://"
		 + currentUrl.getHost() + ((currentUrl.getPort()== 80) ? "" :
		(":" + currentUrl.getPort())) + "/portal/server.pt/gateway/PTARGS_0_0_";
		
		String gatewayEnd = "_0_0_43/";
		
		return gatewayBase + Integer.toString(portletId) + gatewayEnd + CreateGatewayFriendlyURL(docUrl);
	}
}

Happy gatewaying!

What's a common quality of most developers and system administrators? Laziness. I am, unfortunately, no exception to this rule. It is because of this laziness I decided to see if it would be possible to dual boot an ALUI portal instance on my laptop. Let me step back...

What do you mean Dual Booting?

What I mean by dual booting the portal is being able to run (perhaps simultaneously) 4 different common configurations of the portal foundation:

• a .NET/IIS instance
• a Java/Weblogic/Tomcat instance
• an Oracle XE/10g instance
• a SQL Server instance

Ideally, this is done with minimal configuration and/or hassle. Of course getting all of the environments working means modifying some of the config files, so running all of the environments at the same time would take some screwing around with environment variables and install directories that I am just too lazy to do.

This can be useful for a number of different reasons. In my case it can become a necessity if I'm transitioning from a client that wants me to develop using the server API in .NET to a client running on Solaris who wants a Java-based automation server script. Maybe this isn't a 'normal' scenario for you, but maybe you want to evaluate different capabilities of the product or just play around. Either way, I thought my laziness might make an interesting article.

How do I do It?

Luckily, it doesn't take much. Since many of the portal's back-end services were designed with modularity in mind, as long as you know how to play with the right config settings you are probably almost there. This is not to say that this is either a supported configuration, or something you should consider doing in a production environment. Here's what I did:

1. Create an oracle SID and .NET database to house portal components. I used Oracle XE, so there was only one SID for me to use. Make sure both the user name and password for the plumdbuser are identical for both instances.
2. Install image server components to a static web server (apache, IIS). Optionally, create a DNS alias for the image server to allow easy switching (I also use an alias so I can host from different locations on a whim).
3. Install the .NET version of the portal (all components).
4. Install the Java version of the portal to the same directory  -- just the administrative server (use either database configuration).
5. Optionally install a Tomcat/Weblogic portal veresion as well (you can also just deploy the portal.war in multiple places, if this suits you).
6. Script the Oracle and SQL Server databases with the appropriate scripts in $PORTAL_HOME/sql.
7. Copy the jar files from the portal.war (use a zip file editor -- I prefer Winrar) to $PORTAL_HOME\webapp\portal\web\bin.
8. Modify $PT_HOME/settings/portal/portalconfig.xml in the following way:

• Set the WebHome value to $PORTAL_HOME/webapp/portal/web/bin
• Ensure database-connection:adonet-license-file-directory is set to $PORTAL_HOME/bin/assemblies in serverconfig.xml (this may be wrong if you installed .NET first).

At this point you have a base configuration which can be modified to support any web application server or database configuration.

To switch between weblogic/apache/tomcat/IIS:

• Modify AdminSiteBaseURL in $PT_HOME/settings/portal/portalconfig.xml to point to the appropriate port, restart appropriate web server

To support Oracle/SQLServer:

• Modify $PT_HOME/settings/common/serverconfig.xml by changing the following in the database component:
  - database-connection-port value should be set to 1433, 1521 or other port
  - database-connection:dbtype value should be set to either oracle or mssql
  - if not using Oracle, comment out database-connection:dbname

If any custom components have been installed in one environment, it may be necessary to edit the Custom*.xml files to add/remove libraries.

Of course, there are some things which may seem a bit 'off'. If you look in the server settings in your Administrative Utility dropdown, you may see some wacky URL's. On the whole, though, you should be able to build, test and compile Server API code for any platform, and probably run most of the embedded application servers as well.

A follow up question many of you may have is why I don't just run a VM image of the environment I want (many of my colleagues do this). The answer is twofold: 1.) a VM provides an annoying performance lag, and 2.) that would mean I would have to actually move my source code/binaries around (remember the first sentence of this article?).

This project has been slowly gathering momentum on my laptop, as both my notes and stories at clients sites fill up my hard drive. I think it's about time someone shed some light on the 'secret' part of the product stack: the embedded application server. Sure, we could treat the server as a 'black box', but if you're like me you want to reach your hand inside and feel around a bit to see what's there. I'm pretty sure Schrodinger wouldn't have wanted me hanging around his cat. It seems funny that something which can both single handedly make or break your portal performance numbers and has so much room for configuration seems to have received so little attention in the BEA documentation.

In order to address some of the questions I generally get in the field, as well as earn some ALUI street cred, I'll try to give a decent picture of what actually happens under the covers when you start up almost any ALUI product, and what you can do to manipulate that startup. Oh yeah, I guess now would be a good time to mention now that none of this should be taken as an official recommendation from BEA. Proceed at your own risk.

ALUI Application Arichtecture

Below you'll find a very rough outline of how the ALUI embedded application server works. In the rest of this post, I'll deconstruct the application server piece by piece. For specific examples, I'm going to choose the ALUI API Service (located in $ALUI_HOME/ptws/). Also, since this example is taken from my WinXP laptop, some of the terminology will be in Microsoft-ese. Translating it to a Solaris/Linux environment should be a snap.

The Tanuki Software Wrapper

At the end of the day, the initial bootstrap for any ALUI embedded application server is the Tanuki Software application wrapper. The wrapper itself is an open source java service wrapper that allows application developers to rapidly embed logging, support for services, and some other features into their applications. Knowing this, of course, we can start investigating the Tanuki Software website for more information on the wrapper: http://wrapper.tanukisoftware.org/.

We can use what we know about the wrapper to establish a foothold in the mountain of configuration files in our alui directory. Looking at the apiserviced.bat file yields the wrapper executable, as well as the location of the wrapper configuration files. In the case of the ws server, we're looking at the following:

set _WRAPPER_CONF="%_REALPATH%..\settings\config\wrapper.conf

Thus, the bulk of the wrapper's startup configuration comes from the wrapper.conf file, and if we look at the bottom line of wrapper.conf, the wrapper_base.conf file (also found in the settings/config directory of the API web service.

Using this knowledge, we'll look at some of the wrapper's configuration settings to figure out what's going on and how we can manipulate it to our advantage.

Wrapper Logging

The first thing any good administrator will wonder about an application is: how can I figure out what it's doing? In the case of the wrapper, we can amp up the its log settings by changing the wrapper.console.loglevel, wrapper.logfile.loglevel, and wrapper.syslog.loglevel configuration parameters (valid settings for these parameters are enumerated in the file). What this will do is allow us to see what the console and error output of the embedded application server is, as well as monitor what happens to the wrapper itself. Unfortunately, we won't be able to see much more than that, since the loglevel settings only control the wrapper itself, not the embedded application.

That said, startup problems with the embedded application server, as well as the wrapper's JVM running periodic pings to make sure the application is still running, will all show up here. Cranking the loglevels to DEBUG/INFO and doing a tail -f on the log file specified by wrapper.logfile during startup is always a good idea.

Which brings us to controlling logfile location and sizes. As I stated, wrapper.logfile will allow you to control where logfiles are created. This, along with the loglevel settings, wrapper.logfile.maxsize, and wrapper.logfile.maxfiles should allow you to keep your logs under control (again, see comments in the batch file for more detail).

But Where is the Server?

You may have noticed that we have a whole bunch of configuration files for each product, but no server. What gives? You may have noticed some lines in the wrapper_base.conf file that start like this:

%COMMON_PATH%/container/

That path can usually be found in the $ALUI_HOME/common directory. As you will see if you look at this directory on a few ALUI servers, there is more than one embedded application server. Most products actually use Tomcat, but Publisher uses JBoss (again, see the config files for more details). For the purposes of this post, I'm going to shy away from JBoss configuration and focus on Tomcat.

ALUI's Tomcat Bootstrap

Our next stop on this tour de force of the wrapper configuration is the wrapper.app.additional parameters located in the wrapper_base.conf file. These parameters are the most important of all the wrapper parameters, since they are the arguments passed to the java instance executing the application server. In essence, they determine what program will actually be launched by the application server. These parameters are listed sequentially, with a number appended to each additional argument.

Those familiar with the ALUI portal products for any length of time will know that they came from a little company called Plumtree Software. Back in the day, *everything* was called Plumtree. Nowadays, we only get to see plumtree branding when we look under the covers. I bring this up because you'll notice a bit of Plumtree branding in application parameter 1: com.plumtree.container.Bootstrap.

What does this do? Rather than go right out and launch an application server with the Tanukisoft wrapper, ALUI developers created their own boostrap class to handle the heavy lifting for us. One of the commented arguments you'll notice in the batch file is the original Tomcat startup bootstrap: org.apache.catalina.startup.Bootstrap.

What this bootstrap really does, that the Tomcat bootstrap does not, is read a very simple configuration file and generate a server.xml instance to pass to the Tomcat bootstrap (for those not familiar with Tomcat, server.xml is the configuration file used to configure Tomcat). In other words, Tomcat has been dumbed down a bit. The simple configuration file I'm talking about is actually set up as a java system property in the wrapper.java.additional parameters specified in wrapper_base.conf:

wrapper.java.additional.5=-Dplumtree.application="%APPLICATION_PATH%"
wrapper.java.additional.6=-Dplumtree.application.config="%APPLICATION_PATH%/settings/config"

These parameters tell the bootstrap loader where to look for its configuration. The loader then grabs application.xml, parses it to create a server.xml file, writes the server.xml to the same directory as application.xml and starts the tomcat bootstrap with a parameter indicating where the server.xml file is located. If you want to see this in action, delete any server.xml in the application.xml directory and restart your ALUI application of choice. Magically, a new server.xml will appear (provided the wrapper started up okay).

Why this is good: It's simple. All you have to do to configure the application server is edit application.xml and restart the server.

Why this is not so good: Now we are locked into a server.xml configuration for our Tomcat instance. Every time we restart, we are forced to accept the default generated server.xml with a few minor customizations.

What if, for example, we want to bind our Tomcat server to a specific network adapter, rather than the same port across all IP's (for instance, if there is a port conflict with another application)? Tomcat allows this functionality, but the way the application server works, this particular scenario would seem all but impossible to accommodate.

Of course, if that were true then why would I have mentioned it? By being a little sneaky, and knowing a bit about how the bootstrap loader works, we can create and customize our own server.xml. Before you go about this, remember my at your own risk provision at the top of this post.

In order to customize, we first have to start the application server once, so that we can work with a 'stock' server.xml. This is the one that has the big ***THIS FILE IS AUTOMATICALLY CREATED DO NOT MODIFY IT***. Did I mention my disclaimer at the top of this article?

Once we have this file generated, we have to figure out two things:

1. How do we stop the bootstrap from generating this file?
2. How do we tell the application server where our modified server.xml is?

First let's deal with #1: preventing the bootstrap loader from generating the server.xml. It turns out this is surprisingly easy. Simply don't tell the bootstrap where the application settings are. In other words, just leave off the last java additional parameter. In our example, I took out the following line:

wrapper.java.additional.6=-Dplumtree.application.config="%APPLICATION_PATH%/settings/config"

Now for the second part of the equation: telling tomcat where to find our modified server configuration. Since this is Tomcat, it turns out to be a snap as well. All we have to do is change our wrapper_base.conf arguments to pass the server.xml as a configuration parameter, in much the same way the bootstrap loader would have. Here are the relevant lines from my wrapper_base.conf file:

wrapper.app.parameter.1=com.plumtree.container.Bootstrap
wrapper.app.parameter.2=3
wrapper.app.parameter.3=-config
wrapper.app.parameter.4=%APPLICATION_PATH%/settings/config/server.xml
wrapper.app.parameter.5=start
wrapper.app.parameter.6=org.apache.catalina.startup.Bootstrap
wrapper.app.parameter.7=true
wrapper.app.parameter.8=1
wrapper.app.parameter.9=stop

Notice a few things here:

• The -config parameter is passed as the third argument, the actual configuration location is the fourth.
• I had to modify parameter 2, which tells the Tanuki Software wrapper how many arguments I'm passing to the bootstrap (it was 1 in the original file).

That's it. Our next step is to test the configuration by changing changing something in server.xml, leaving application.xml alone and seeing what happens when we start the application server. I chose to change the port attribute of the Connector element. After I started my application server I simply checked for the server listening on the new port (11900 in my case) by running:

netstat -a | grep 11900 (if you don't have grep, you can try telnet localhost 11900)

My results were satisfactory:

TCP xxxxxxxx:11900 0.0.0.0:0 LISTENING

Solaris Users, Read On...

As a consequence of this blog post, I was fortunate enough to get an opportunity to try this on Solaris. Unfortunately, my results were a little less satisfactory than with the Windows version of the embedded application server. It turns out there is a nasty check based on the os.name java system property that attempts to remove quote characters from all paths passed to the bootstrap. Unfortunately, the check can't handle null strings. This means that leaving out wrapper.java.additional.6 is not really an option.

Thanks to some dialog with Brian Hak, and a bit of playing around, there is a solution. Simply leave the configuration as-is and pass an additional server config to the bootstrap (with a different path than the server.xml that is generated automatically). The catch here is that you must pass the -config argument before the start flag in the wrapper.additional parameters (otherwise you'll get classpath errors). In a nutshell, wrapper.java.additional doesn't change from the original, but wrapper.app.parameters now become:

wrapper.app.parameter.1=com.plumtree.container.Bootstrap
wrapper.app.parameter.2=3
wrapper.app.parameter.3=-config
wrapper.app.parameter.4=%APPLICATION_PATH%/settings/config/newServer.xml
wrapper.app.parameter.5=start
wrapper.app.parameter.6=org.apache.catalina.startup.Bootstrap
wrapper.app.parameter.7=true
wrapper.app.parameter.8=1
wrapper.app.parameter.9=stop

Using the Wrapper to Tune the JVM

At one point in this article I mentioned that the embedded application server was also a gold mine of performance improvement. I could fill an entire post up with Tomcat performance tuning advice, but I think it might be more beneficial just to mention a few things to follow up with here. More specifically, there are a few arguments we can pass to the JVM, using the wrapper.java.additional arguments which can help us tune its garbage collection to optimal performance. More specifically, we can do things like log garbage collection via -Xloggc, add the -server flag to our JVM, etc.. For more information, here's an interesting article about Java Garbage Collection Tuning.

Hints, Allegations and Things Left Unsaid

As you can tell, there is a lot we can do with the embedded application server that isn't described in the instruction manual. As such, here are a few more things to think about before I sign off: it might be possible to change the entire embedded application server, JVM (major or minor revision), or even host out of a non-embedded environment. These are all exercises that, of course, would be unsupported by your friendly BEA helpdesk, but they may be of interest for the more enterprising, or more curious, reader.