Introduction

At some point of our Zope life, we need to incorporate some pages that are not in zope-land; for example, some data in php, or perl. Or even some data that's not accessable through RSS/RDF.

That's where KebasData comes in.

What is it?

KebasData is a Zope product that fetches/grabs data from some other location or server. We can then call the KebasData instance to render the acquired data.

Instantiating KebasData will give us the ability to get a url, parse for particular pattern, and render or view it. We can set the KebasData freshness to some number so that a call to KebasData will refresh if the required time has passed.

KebasData attributes

1. id: the necessary id

2. title: the title, of course

3. pattern: the pattern to look for in the page.

4. start pattern: The start of our file. If left blank, the whole page will be parsed. If specified with end pattern, that particular data will be parsed.

   This will hopefully eliminate unnecessary data.

5. end pattern: The end of data to be parsed.

6. render method: The default render method. Used when accessing directly thru URL e.g. http://site/kebasdata_id/. also used when calling from another method, e.g <dtml-var "kebasdata_id.view()">

7. refresh: Freshness in seconds.

8. request method: either GET or POSt

9. parameter list: used with POST. We can also pass the parameter list as query string.

How?

Let's look at a few examples.

UserFriendly in your page

Say we want to put UserFriendly in our page.

1. Instantiate KebasData

2. Fill in the following:
   

   id:  userfriendly
   title:  UserFriendly
   url:     http://userfriendly.org/static/
   pattern: <img ALT="Latest Strip".*?>
   start pattern:
   end pattern: 
   request method: GET
   

3. The refresh time and render method is available in the Edit View.

4. Create your render method, which for this example is a dtml-method we'll name user_html.

5. In user_html enter the following:

   <dtml-var
   "match[0]">

6. To use Script Python as your render_method, do as:

   #Parameter list: a, b
   return context.match[0]
   

   
   Note: the parameter list contains two place holders for mappings passed (_.None, _) or (context, context.REQUEST)

7. Now, a call to http://server.com/userfriendly will get the page, parse for the pattern, and return a list of matches.

   The match in this case is

   <IMG ALT="Latest Strip" height=219 width=576 BORDER=0 SRC="http://www.userfriendly.org/cartoons/archives/somedate/somegif.gif">

   And viewing this will show the UserFriendly comic strip for that day.

8. To call it, use <dtml-var "userfriendly.view()">

The render method is a DTML Method, or python script that does the default KebasData rendering. We can do anything we wanted here, string.strip or string.replace the KebasData matches, depending on our needs. In this example, there was only a single match, so the method was quite simple. (A slightly more complex example is given below.)

Refresh is in seconds. When a user accesses the KebasData instance, KebasData will first check its freshness. If it's still fresh, no call to the url is done. Else, KebasData will call the url, parse, and get the matches.

Zope News

Let's also say that we're interested in Zope News. There exist a RSS/RDS file taht we can download. However, for this exercise, we assume that there's no such file.

1. Instantiate a KebasData

2. fill in:

   id: zopenews
   url: http://www.zope.org/ZopeNews
   pattern:<p class="headline">(.*?)</p>.*?<a href="(.*?)">
   start pattern: 
   end pattern:
   request method: GET
   

3. Edit the KebasData instance. add a render method

   render method: zopenews_html

4. Now create a DTML Method with the id zopenews_html

    
   <dtml-in "_.range(0,5)">
   <br><a href="<dtml-var "match[_['sequence-item']][1]">"><dtml-var "match[_['sequence-item']][0]"></a>
   </dtml-in>
   

5. Or, a Script (Python) as the render_method:

   #Parameter list: a, b
   for u,t in context.match:
     print u, t
   return printed
   

6. to call it from another method:

   <dtml-var "zopenews.view()">

Since the pattern uses grouping, the match will be a list of tuples. The title is the first item in the tuple, and the URL is the second item.

What we're doing in the DTML Method zopenews_html:

• iterate through the list of tuples

• since this is a tuple, we need to do the necessary to get the proper item

• When render_method is a Script(Python), things are much better and comprehensable. Notice that match contains only patterns we enclosed in ()

The Properties and Test Views are great for troubleshooting our patterns.

Using Google as site search engine

This is an exercise to using KebasData dynamically, and is just that, an example. :)

We will have a search form that will search www.zope.org with google, and return only 100 relevant matches. All the while, remaining in our site.

• Instantiate a KebasData. Call it goo

• Enter these in the necesary fields. You may need to go to Edit view to cadd render method.

  id: goo
  title: google
  pattern:      about <b>(.*?)</b>|<p>(.*?) - <A href=/search
  start pattern:
  end pattern:
  render method: result
  refresh: 1
  request method: GET
  parameter list:
  

• Add a DTML Method with id: result

  <dtml-try>
  <dtml-call "REQUEST.set('ada',goo.match[0][0])">
  
  <dtml-if "REQUEST['ada'].find(',') > -1">
  <dtml-call "REQUEST.set('ada',REQUEST['ada'].replace(',',''))">
  </dtml-if>
  
  <dtml-if "_.int(REQUEST['ada']) > 100">
  Only 100 of <dtml-var "goo.match[0][0]"> listed.
  <dtml-else>
  <dtml-var "REQUEST['ada']"> listed.
  </dtml-if>
  <dtml-in "_.range(0,_.len(goo.match))">
  <dtml-let i="_['sequence-item']">
  <br><dtml-var "goo.match[i][1]"></font>
  </dtml-let>
  </dtml-in>
  <p><dtml-in "_.range(25,100,25)">
  <dtml-let lvlchk="_['sequence-item']">
  <dtml-if "_.int(lvlchk) < _.int(REQUEST['ada'])">
  <a href="srch?srcid=<dtml-var "REQUEST['srcid']">&start=<dtml-var lvlchk>"><dtml-var lvlchk></a> |
  </dtml-if>
  </dtml-let>
  </dtml-in>
  <dtml-except>
  Not Found
  </dtml-try>
  

• Add a DTML Method for the search form with id search:

  <html><body>
  <form action=srch method=post>
  search: <input type=text name=srcid>
  <br><input type=submit>
  </form>
  </body></html>
  

• Add a DTML Method with id srch

  <html><body>
  <dtml-if start>
  <dtml-call "REQUEST.set('srchterm','http://www.google.com/search?num=25&q=site%3Awww.zope.org%20' + t(REQUEST['srcid']) 
  +'&start=' + _.str(start))">
  <dtml-else>
  <dtml-call "REQUEST.set('srchterm','http://www.google.com/search?num=25&q=site%3Awww.zope.org%20' + t(REQUEST['srcid']))">
  </dtml-if>
  <dtml-call "goo.get_matched(url=REQUEST['srchterm'])">
  <dtml-var "goo.view()">
  
  </body></html>
  

• Add a Script (Python) with id t. This is to encode the query string.

  from Products.PythonScripts.standard import url_quote_plus
  return url_quote_plus(srcid)
  

Now, point our browser to the search form, and we should be in business.

In this example, we're generating the url dynamically by calling KebasData get_matched.

If there's some permission problems, just make sure the intended roles (Anonymous User or Local roles) have "Access Kebas Data get_matched" enabled.

These are not really tested yet, so expect some bugs. If you do encounter them, mail me the fixes.

Conclusion

Integrating non-Zope site in Zope sites is doable. There are lots more that can be done with KebasData. If you do have examples on how you use KebasData, do mail them to me so that i can include it here.

hope this will be of help to somebody. If it is, i don't mind getting a mail telling me that you're using KebasData.

Demo of all the examples.