Advanced DTML Techniques
Created by .
Last modified on 2003/08/01.
Introduction
Document Template Markup Language (DTML) is a central component of
Zope. DTML functions both as an HTML formatting language as well as
a simple scripting tool.
This document tries to explain some of the more advanced concepts,
features and gotchas of DTML. This document does not replace the DTML User's Guide. The Guide
is the definitive reference to DTML. If you haven't read it at least
twice, do so now. This document is more aimed at explaining some of
the ideas and quirks behind DTML, rather than spelling out exactly
how to use it.
Documents versus Methods
DTML primarily displays Zope objects in HTML. This is the driving idea behind DTML Methods--they are methods which allow Zope objects to represent themselves textually.
DTML is also used to hold textual content which can be viewed as web pages. This is the main idea behind DTML Documents.
As we will see later there are technical differences between DTML Documents and Methods that make them suited for these different purposes.
Namespaces
The most important service DTML provides is variable lookup services. How is this done? With namespaces.
Like Python, DTML uses multiple namespaces to find a variable when it is referenced. Unlike Python, DTML has a potentially many heterogeneous namespaces to consult rather than a fixed number of well defined namespaces. This complexity makes DTML somewhat confusing, but it also makes it quite powerful.
DTML uses a stack of namespaces to lookup variables. First Zope looks for the variable in the top namespace, then in the next one down and so on until it finds the variable.
Depending on how you call a DTML Document or Method, different namespaces will be placed in the stack.
At different points while rending a DTML document, namespaces can be pushed on, or popped off the namespace stack. That's why:
<!--#var title-->
doesn't always refer to the same title at different parts of a DTML object.
Sounds confusing, and it is. But you probably already have an implicit understanding of this operation. For example, let's take a snippet of DTML:
folder's title: <!--#var title-->
<!--#in objectValues('DTML Document')-->
sub-document's title: <!--#var title-->
<!--#/in-->
Suppose this example was in a DTML Method of a Folder entitled "My Folder". Suppose "My Folder" contains two DTML Documents entitled "Doc 1" and "Doc 2". Then this DTML snippet would render this text:
folder's title: My Folder
sub-document's title: Doc 1
sub-document's title: Doc 2
This seems fairly clear. But if you look closely you'll see that the in tag manipulates the namespace stack so that each time through the in loop the title variable refers to a different title. What the in tag does is push a namespace on the stack at the beginning of each loop and pop it off at the end. The namespace it uses is built from the current sequence-item.
Namespaces and the with tag
The with tag does the same sort of namespace manipulation as the in tag, only it is more explicit about it. The with tag pushes a given namespace onto the stack when Zope comes to the opening with tag, and pops it off the stack when Zope reaches the closing with tag.
This explicit namespace management can be useful in many situations. For example suppose you have a form which contains an input named "title". When you submit the form, the DTML Method that handles the form may want to refer to this title. However, simply referring to the title may not work:
<!--#var title-->
This will find the title of the enclosing Folder, rather than the title in the request's form, since the enclosing Folder's namespace is on top of the namespace stack by default. One way to solve this problem is to use a with tag to explicitly push the request's namespace on top of the namespace stack:
<!--#with REQUEST-->
<!--#var title-->
<!--#/with-->
DTML calling arguments
A DTML object's inital namespace stack is set by its calling arguments.
DTML objects can be called from Python with three arguments.
- a client object
- a mapping
- optional keyword arguments
When a DTML object is called, these arguments are pushed onto the DTML object's namespace stack with the keyword arguments on top, followed by the clients, followed by the mapping.
Now, what does this have to do with Zope? Simple, when Zope's ORB publishes a DTML Document or Method it passes the client and mapping arguments. The client is the enclosing (or acquiring) Folder. The mapping argument is the HTTP request.
Note that, in the case of DTML Documents, the Document itself is used for variable lookup along with and the client argument.
The upshot is that by default the namespace stack has the request below the enclosing Folder. That's why we need the explicit with tag to find our form's title.
Types of namespaces
So far we haven't really explained what a namespace is. We know it's something you can look up variables in. In Python, namespaces are essentially mapping objects which hold the variables defined in classes, instances, and modules. In DTML namespaces operate in much the same way. In general, any object or mapping can act as a namespace. Variables are looked up in objects by finding object attributes and methods. Variables are looked up in mappings normally. However, Zope constrains what you can lookup in a DTML method. See the section on security for more information on lookup constraints.
Idle reflections
Let's take a breather here and reflect a little. We've seen that DTML uses a dynamic stack of namespaces. Often these namespaces correspond to objects. So we have a scenario in which variables are looked up by traversing layers of objects. This motif is quite central to Zope. Both object publishing and acquisition echo this theme of object traversal. The Zope ORB works downward through the object hierarchy examining objects until it finds the requested object. Acquisition works upward through the object hierarchy binding methods and attributes to called objects. DTML namespaces similarly form a dynamic stack of objects which are consulted in order to find template variables.
The DTML namespace stack differs from object publishing and acquisition in that it is not tied to the object hierarchy. The stack is constructed initially by calling a DTML object, and then is modified dynamically in the body of the template.
Indirect variable lookup
Just as you can use Python's getattr and __dict__ in Python to lookup names indirectly, so too in DTML you can lookup variables indirectly.
Indirect lookup is just a question of consulting the namespace and asking it if it has a variable. In DTML the way to do this is with the special variable whose name is underscore. The underscore variable has a getitem method, and also can be treated like a mapping to access the namespace's variables. So for example:
<!--#var foo-->
<!--#var "_['foo']"-->
<!--#var "_.getitem('foo')"-->
All do pretty much the same thing. One subtle difference between these three forms is that the first and second form "render" the resulting variable, while the third does not. We'll discuss rendering later. Just for completeness we should mention that the getitem method takes and optional second argument which indicates whether to render or not. By default it does not render.
Since indirection is confusing, you should only use it when necessary. For example, there is really no reason to write:
<!--#var "_['foo']"-->
when you can more simply write:
<!--#var foo-->
If you know that you're interested in foo, just ask for it. There's no need to say "give me the variable whose name is 'foo'", when you can just say "give me 'foo'".
Normally you will use indirection to access variables whose names you don't know when you are coding the DTML. For example:
The value of <!--#var interesting_attribute-->
is <!--#var "_[interesting_attribute]"-->
Notice that there are no quotes around interesting_attribute in the second var tag. That's because we're not looking for the variable named interesting_attribute, (in which case we could just write <!--#var interesting_variable-->). What we want instead is the variable whose name is given by the variable interesting_variable. Whew!
How does this work? Suppose the above DTML fragment was in a DTML Document entitled "My Doc". Then if you called the Document with a form that assigned interesting_attribute to "title", then the Document would return:
The value of title
is My Doc
Declaring variables
What if you want to declare variables inside a DTML object, how can you do it? Well there are a couple ways. The most common way to declare a variable which can be used later in the template, but which goes away when the template has finished rendering is to set a variable in the REQUEST object. This is done with the REQUEST object's set method:
<!--#call "REQUEST.set('food','pork chops')"-->
Now there is a food variable in the REQUEST object's namespace and it can be looked up normally.
Another way to do the same thing is to use the with tag. You can use the with tag to push a mostly empty namespace on the stack, which only contains some variables which you set. This is done by using the underscore's namespace method. For example:
<!--#with "_.namespace(food='pork chops')"-->
<!--#var food-->
<!--#/with-->
As you can see, the namespace method takes keyword arguments which set variables in the namespace.
You should note that using the with tag puts your variables on top, so they'll be sure to be found when looked up. Also, your variables will disappear at the close of the with tag. Setting a variable in the REQUEST makes your variable subject to the visibility of the REQUEST object's namespace. However, variables set in the REQUEST are persistent through out the entire template.
New in Zope 2.0 is the let tag which allows you to define namespaces easily.
For example:
<!--#let food="'pork'" name="AUTHENTICATED_USER.getUserName()"-->
<!--#var name--> eats <!--#var food-->
<!--#/let-->
This is quite similar to the with..namespace construct. Variables defined in the let tag can either be rendered normally of variable expressions, if they are quoted.
Acquisition
So far we have neglected to mention the importance of acquisition in DTML. Acquisition has two major effects with respect to DTML: objects can acquire DTML Methods, and DTML namespaces corresponding to objects can use acquisition to lookup variables. Let's take each effect and look at the subtleties.
One of Zope's main cool technologies is acquisition, and acquisition of DTML Methods by objects is a prime example of this power. As we saw earlier, when a DTML Method is called, its enclosing or acquiring Folder is passed as its client namespace. This means that if your DTML method refers to a variable named "title", this variable is first looked up in the Folder's namespace. This allows DTML Methods to act like templates, tailoring their content to the attributes of the object which acquires them.
Another effect of acquisition is felt in the variable lookup process. For example:
<!--#with Foo-->
<!--#var Bar-->
<!--#/with-->
In this DTML fragment the variable Bar is looked up in Foo. Even if Foo does not have a Bar attribute, Bar will be found in the Foo namespace if the Foo object can acquire Bar. This effect can catch you unaware if you are not careful.
Variable rendering
The DTML var tag does not merely lookup variables, but it "renders" them. The process of rendering is not just coercing objects into strings, but calling objects if they are callable, and handling DTML objects specially. Zope's rendering rules are designed to do what Zope thinks is the right way to display an object. However, you may not always want to let Zope render and object automatically.
For example, Zope normally renders DTML objects referred to in other DTML objects by calling them the with current DTML namespace stack. However, you may wish to pass a different namespace. For example, suppose you have a DTML method defined at the top level which displays a site map. This method relies on attributes and methods of the top level Folder to display its site map. If you refer to this method in a DTML Method deep in your site, Zope will pass it a namespace which doesn't have the top level Folder on top of the namespace stack. Instead you may wish to manually render the method by calling it with the top level Folder as a client, and the REQUEST as a mapping:
<!--#var "site_map(PARENTS[-1],REQUEST)"-->
This works because the PARENTS variable is a list of parent objects. So PARENTS[-1], which is the last item in the list, refers to the top level Folder.
You could achieve similar results using the with tag:
<!--#with "PARENTS[-1]"-->
<!--#var site_map-->
<!--#/with-->
While they both achieve the same results, there is a subtle difference between these two methods. In the first example, the namespace stack contains two items, PARENTS[-1] and the REQUEST. In the second case it contains an indeterminant number of items, with PARENTS[-1] on top.
If you wish to get a variable without rendering it, one way is to use a variable expression. For example,
<!--#var expr="parrot"-->
differs from, <!--#var parrot-->
in that it does not render the variable. Don't forget that if you wish to get a variable indirectly without rendering it, use getitem. For example:
<!--#var expr="_.getitem(bird_type)"-->
retrieves that variable named by the variable bird_type without rendering it. By contrast:
<!--#var expr="_[bird_type]"-->
renders its results.
Variable expressions
As we've seen, you can use the var tag to not only render variables, but to spell out primitive Pythonish expressions. This facility allows you to supply arguments when you call variables, and to generally interact with the Zope object system. All sorts of Zope objects provide all sorts of interesting services which you can access via DTML expressions.
One way to find out about the services available to you through DTML is to consult the Help tab in the management interface. The object reference which is available there shows you the publicly accessible methods of different classes of Zope objects. This is quite useful, and guaranteed to be up to date, since it relies on the actual code to generate the reference. One thing to keep in mind is that any given Zope object will have more methods available than those listed in the Object Reference. Why? Because objects acquire methods and attributes from their parents.
Names are looked up in variable expressions in the same way they are normally in DTML. So to write an express which evaluates to foo plus two simply write:
foo plus two equals: <!--#var "foo + 2"-->
There's no need to write DTML inside DTML. In fact you can't, as this common mistake illustrates:
this is an error: <!--#var "<!--#var foo--> + 2"-->
The special underscore variable gives you access to many useful Python functions which you may want to use. See the DTML User's Guide for a complete list of available functions.
Why not simply write normal Python instead of variable expressions? The reason is that variable expressions provide many security services which vanilla Python doesn't. For example, variable expressions provide protection against endless loops, and many kinds of denial of service attacks. DTML expressions also limit visibility into the Zope object system as we will soon see.
Security
DTML gives you a window on Zope's object system. However, it is a limited view. You can only access Zope objects whose names do not begin with an underscore, and objects which you have adequate permissions to access.
To get around the permissions barrier, Zope provides Proxy Roles for DTML Documents and Methods. Proxy Roles are similar in spirit to the Unix setuid facility. For example, a DTML Method with a "Manager" Proxy Role could perform restricted management functions such as deleting Zope objects even when called by a user without the "Manager" role.
Knowing when to say when
No discussion of advanced DTML would be complete without mentioning External Methods. DTML can get pretty complex. When you find yourself setting lots of variables, and using a lot of variable expressions, it may be time to consider using an External Method instead. Often it is much easier and clearer to express some computation in Python than it is to express it in DTML.
Another important tool for fighting complexity is to refactor templates into a number of sub-templates. Finding the right way to factor your problems into templates, properties, and external methods is an art. Luckily Zope is forgiving, and lets you experiment quite a bit.
Conclusion
DTML can be complex. One of the most important way to keep track of what's going on is to follow the namespace stack. Understanding how it works greatly improves your ability to make DTML serve your needs.
Good luck and have fun!
Log in
Forgot your password?