Zope Changes This file contains change information for the current Zope release. Change information for previous versions of Zope can be found in the file HISTORY.txt. after Zope 2.5.1 new Features: - Added support for gzip content compression for clients that support it. See lib/python/ZPublisher/HTTPResponse.py for more details. - Added ZCTextIndex plug-in index product. A replacement for TextIndex. - Removed the venerable but senile QuickStart folder from the default FileStorage. "Alas, poor Yorick! I knew him, Horatio." - Signal handling and log rotation All Zope process will respond to signals in the specified manner: SIGHUP - close open database connections and sockets, then restart the process SIGTERM - close open database connections and sockets, then shut down. SIGINT - same as SIGTERM SIGUSR2 - rotate all Zope log files (z2.log, event log, detailed log) The common idiom for doing automated logfile rotation will become: kill -USR2 `cat /path/to/var/z2.pid` The common idiom for doing "prophylactic" restarts will become: kill -HUP `cat /path/to/var/z2.pid` When a process is interrupted via ctrl-C or via a TERM signal (INT, TERM), all open database connections and sockets will be closed before the process dies. This will speed up restart time for sites that use a FileStorage as its index will be written to the filesystem before shutdown. Unspecified signals kill the process without doing cleanup. - ZCatalog no longer has a hand in managing text index vocabularies. The cruft associated with this functionality has been exorcised. No default indexes or metadata elements are defined for you when you create a new ZCatalog. Since we now have many new kinds of plug-in indexes it no longer made sense to do this anymore. - A new permission "Copy or Move" was added. This permission may be used respective to an object to prevent objects from being copyable or movable while within the management interface. The "old" behavior stipulated that users whom possessed the "View management screens" permission to an object's container could copy or move the object arbitrarily, even if they had limited access to the object itself. Once the object was moved or copied, the user became the owner of the new object, allowing them to see potentially sensitive information in the management interface for the object itself. This permission is granted to Manager and Anonymous by default, and must be revoked on an object-by-object basis if site managers intend to provide management screen access to folders which contain sensitive subobjects. This patch came as a result of Collector #376 (thanks to Chris Deckard). - Structured Text's "DocumentWithImages" class did not recognize image filenames with underscores. - The getElementsByTagName method of STDOM (used by Structured Text) would croak on most documents, especially those containing unwrapped text nodes. Fixed. - FileUpload objects now evaluate false when the have an empty file name. Making it easier to check for omitted file upload form fields. - ZClasses now use a python script as their constructor method instead of a DTML method. Also, ZClasses inherit from CatalogPathAwareness now instead of CatalogAwareness. - added browser_default hook to ZPublisher. This allows objects to specify the path to the default method that the publisher calls when the object is published. The default for objects not defining browser_default is still 'index_html' for bw compatibility. A ZMI configurable browser_default implementation has been added to ObjectManager. You can configure browser_default for OMs via a new "settings" management tab. - added TopicIndexes: a TopicIndex is a container for so-called FilteredSet. A FilteredSet consists of an expression and a set of internal ZCatalog document identifiers that represent a pre-calculated result list for performance reasons. Instead of executing the same query on a ZCatalog multiple times it is much faster to use a TopicIndex instead. - requestprofiler: added new --daysago option and added support for reading gzipped detailed logfiles - DateTime: new functions JulianDay() and week() to perform calculation of the week number based on the Julian calendar. - WebDAV: the new environment variable WEBDAV_SOURCE_PORT_CLIENTS enables retrieval of the document source for dedicated WebDAV clients (see ENVIRONMENT.txt for usage) - Collector #272: Optimisations for RESPONSE.write - Collector #271: New environment variables are now used to send the access log into syslog. ZSYSLOG_ACCESS, ZSYSLOG_ACCESS_FACILITY, and SYSLOG_ACCESS_SERVER now do the same job as the old environment variables without _ACCESS in their name. Those old environment variables still do the same job of sending the event log to syslog. - When run as a daemon on Unix, Zope will now redirect stdin/stdout/stderr to /dev/null - Nicer formatting for the increasingly tall permissions table. - TextIndex: Enhanced splitter functionality now allows the TextIndex to index numbers, single characters. It is also possible to enable case-sensitive indexing. The new configuration options are available through the addForm of the Vocabulary object. - ICP server support. For more information see http://www.zope.org/Members/htrd/icp/intro - STXNG: added new env. variable STX_DEFAULT_LEVEL to change the default level for elements (see doc/ENVIRONMENT.txt) - Collector #304: several catalog optimisations - New implementation of ZODB object cache. The new implemenation is more likely to keep the size of the object cache close to the target size. This change means that memory consumption may be reduced. Some users will need to increase the default cache size, because a too small setting is more likely to hurt performance than it did in the past. Third-party C extensions that use the persistence API must be recompiled, and may need to be updated to work correctly with the new cache; see PER_GHOSTIFY(). - The ZODB Connection is now resposible for registering changed objects with the current transaction. - Implementation of RestrictedCreation fishbowl proposal; Product registration can now include a function used to determine whether that product constructor want to allow objects to be created in the specified container object. - Collector 196: manage_page_style.css is now cacheable. Added freshness information to ImageFile, to improve cacheability of management interface - Collector 358: added a new parameter no_push_item to dtml-in, to inhibit automatically pushing sequence-item onto the namespace stack. - STXNG: Structured Text now supports images by default by using the HTMLWithImages class (has been disabled prior to Zope 2.6) - new option --force-http-connection-close for z2.py to prevent clients from maintaing pipelined connections to the Zope server (Collector #412) - Updated the Interface package to be compatible with Zope 3 Interfaces. This included changing some interface APIs that may affect existing products. - Added a database activity monitoring graph to the control panel, making it easier to tune the ZODB cache size. Bugs: - External methods didn't properly setup func_defaults and func_code when they were first loaded. This meant mapply couldn't properly map arguments on the first try. - Fixed bug #96: Narrower/Wider buttons now work on both CSS and non-CSS compliant browsers. This allows better control for browsers that have a hard time knowing what 100% means. - Fix for Collector #319: filtered_manage_options didn't correctly filter tabs based on permission. - Made repr of an HTTPRequest.record eval'able as a dict (Collector #89). - Fixed bug #144: Upload button on dtml, py scripts, images, files and pts now raises an error if the file is not specified rather than clearing the source. - Fixed bug #275: setPermissionDefault didn't actually set the right permission -> role mappings. - Fixed bug reported on maillist during EWOULDBLOCK when using FTP server (http:// lists.zope.org/pipermail/zope/2002-March/111521.html). - App/FindHomes.py now computes the "real" path for SOFTWARE_HOME and INSTANCE_HOME, resolving any symlinks in any element within paths passed in via the INSTANCE_HOME or SOFTWARE_HOME envvars. Paths that are computed by "dead reckoning" from os.getcwd and module paths are also "realpathed". So for instance, if you use '/home/chrism/Instance' as your INSTANCE_HOME, and '/home/chrism' is a symlink to '/other/home/chrism', your INSTANCE_HOME will be computed as '/other/home/chrism/Instance'. This is necessary to avoid weirdnesses while using "dead reckoning" from INSTANCE_HOME and SOFTWARE_HOME in other parts of the code. POSIX systems only. - Fixed PropertyManager/PropertySheets so that you can safely add a property named 'ids' without breaking your properties page. - Removed spurious 'self' from scarecrow interfaces; updated method-generation in Interface package to ignore self when source is a method (rather than a function). - Collector #32: Use difflib instead of ndiff - Fixed long standing bug in PythonScript where get_size returned the incorrect length. This broke editing using EMACS via FTP or WebDAV. Thanks to John Glavin at South River Technologies for help finding the bug. - Collector #207: fixed problem with inner links in STXNG - Collector #210: HTML() function of StructuredText produced wrong tags. - Collector #166: ObjectManger.all_meta_types() implemented only an incomplete filter based on interfaces. - FTP: Downloading files through FTP has been broken since 2.4.0 because the downloaded file has been stored with a HTTP header at the beginning of the file. Fixed! - FTP: Spaces in usernames inside a FTP file listing are now replaced by underscores to avoid confusion with some FTP clients. - Collector #227: improved handling of unicode string in TextIndex.py with unmodified default encoding in site.py. - Collector #227: z2.py, TextIndex/dtml/manage_vocab.dtml modified to display unicode strings in the vocabulary properly (now using UTF-8 encoding for display purposes) - Collector #250: applied several patches for TextIndex for better unicode support for the GlobbingLexicon - Collector #254: return owner object from getOwner wrapped in its context - Collector #259: walkandscrub.py did not delete all .pyc and .pyo files during installation. Fixed. - Collector #231: BTrees ignoring errors from comparison function - Collector #278: DocumentWithImages could not handle URLs with underscores - Collector #279: changed exception handling for safegmtime() to provide a more intuitive traceback for operating systems with a limited gmtime() implementations - Collector #285: Zope changes its working directory to the var directory at startup - WebDAV: removing an non-existing property now returns a HTTP 200-OK response instead of 404 (now compliant with RFC 2518) - Fixed a bug in TM.py that would cause database adapters to hang on errors in the second phase of the two-phase commit. - Collector #291: ZCatalog not unindexing deleted properties - Collector #266: Retried requests losing track of http request headers, causing Connection:Close requests to stall - Collector #17: Fixed broken links in StandardCacheManagers help - Collector #1: UNIX security fixes: make starting Zope as 'root' secure, stop using 'nobody', warn of insecure umasks - Collector #303: Properties of type 'long' got truncated - Collector #325: adding a new TextIndex to an existing Catalog cleared the standard Vocabulary. - Collector #373: content_type property for Image objects are no longer deletable to prevent malfunction. - Collector #343: The ZCatalogs 'Indexes' view showed the wrong number of indexed objects for FieldIndexes. - FTP server: replaced 'System_Process' by 'Sysproc' to avoid breaking some FTP clients and the output format with overlong usernames. - Fixed a potential bug with cAccessControl's permission role deallocator which would try to decref things which may not have been set, due to a change in the initializer (which will bail out if it doesnt get called with a tuple argument) - Collector #185, 341: PCGIServer and FCGIServer logs corrected and now output extended information like HTTPServer does. - Propertysheets: Ids like 'values' and 'items' are now forbidden as they break WebDAV functionality. Existing Propertysheets are not affected - Collector #348: decapitate() now recognizes both \r\n and \n\n to be compliant with the HTTP RFC - Collector #386: workaround for hanging FTP connections with NcFTP - Collector #419: repaired off-by-1 errors and IndexErrors when slicing BTree-based data structures. For example, an_IIBTree.items()[0:0] had length 1 (should be empty) if the tree wsan't empty. Zope 2.5.1 beta 1 Bugs Fixed - Fixed failed incref in cAccessControl that caused problems when ghosting occured. Zope 2.5.0 Bugs Fixed - Btree module and UnicodeSplitter could potentially fail to incref Py_None on return - Setting proxy roles on DTMLMethods and Documents has been broken - Collector #99: items(), keys() and values() methods of IIBucket objects has been broken when called with min/max parameters. - WebDAV: The long outstanding read-only problem with WinWord and .html is finally solved. Zope sends now an additional ETag header for DTMLMethods and DTMLDocuments. Many thanks to Joachim Schmitz for this hint. - PathIndex: fixed minor bugs, enhanced testsuite - DTML-MIME tag: creates now a 'Mime-Version: 1.0' to make some fussy email programs happy - utilities/requestprofiler - fixed bugs in timed mode which could show nonsensical results. Features Added - TextIndex/Splitters: the constructor of all three splitters has now three new optional parameters: 'maxlen'=(1-256) - to specify the maximum length of splitted words 'singlechar'=(1|0) - allows single characters to be indexed 'indexnumbers'=(1|0)- allows numbers to be indexed The default values of all parameters reflect the standard behaviour. - Enhancements to utilites/requestprofiler.py: Added readstats and writestats features which allow for saves and reuse of profile stats between runs of the requestprofiler. Added urlfocus mode. Urlfocus mode presents a summary of activity within a period of time before and after the invocation of a particular URL as it was recorded in the big M log. This can be useful when trying to track down a problem which you believe is related to some series of URL invocations as opposed to a single URL invocation. - added doc/ENVIRONMENT.txt to document all used environment variables Zope is using - Provided a much more robust tool for recovering data from damaged FileStorage files: Allow recovery of data when: - either transaction or data records are damaged and when - multiple parts of a file are damaged The interface has changed to not modify in place. Other features: - Progress indicator - Verbose output - optional packing - index creation - The last entry in the breadcrumb path in the ZMI is now an underlined hyperlink. - Added a new script "utitilies/check_catalog.py" to perform some consistency checks on a ZCatalog instance - Added the following new products related to sessioning: - browser id manager - session data manager - transient object container - temporary folder - Zope will create default sessioning objects on startup - Zope will create an Examples folder on startup. This folder contains a collection of simple example applications. Bugs Fixed - STXNG: added '+' as allowed character inside URLs - enhanced the checks for 'Domains' field in the UserFolder - Webdav: fixed broken MOVE and COPY commands - Updated DTMLMethod validation support patch with version by Steve Alexander - did not work properly under some acquisition related reaons. - pre-2.5 installation could not properly migrated to 2.5 because of changes in the Splitter API - ZODB conflict errors were logged at a severity that caused tracebacks to be logged to the stupid log file "out of the box". Because conflict errors happen under normal operations, and their appearance in the log frequently alarms folks, typical conflict error logging verbiage has been reduced, and traceback logging has been changed to happen only at BLATHER log level, which means that conflict tracebacks will only be logged if STUPID_LOG_SEVERITY is set to -100 or lower. Zope 2.5 Alpha 2 Bugs Fixed - New accelerated security management bugfix in App/Manager.py, changing calling sequence to positional parameters. - STXNG: re-enabled inner now work also through DTML and not only when using STXNG standalone - STXNG: generated HTML is now more HTML4/XHTML compliant (quoted attributes, inner links) Zope 2.5 Alpha 1 Features Added - re-enabled inner links for STXNG documents. - Zope installations upgraded from pre-2.4 installations did not show the WebDAV LockManager entry in the control panel. - "requestprofiler.py" script in utilities now does better analysis, including: "active" in detailed output is now slightly more meaningful. It is > 0 only if other requests started after it started but before it finished. The greater the active count, the more likely it is that something was going on at the time in which the request ran that caused a slowdown. Multiple files may be analyzed at the same time. Also, script recognizes "U" opcode in big M logs as meaning a restart. - Added a "preview" field to the edit form for Image objects. This provides a preview of the actual image data on the form to make it easier to know what you are working on. The preview image is scaled if necessary to be useful without being obtrusive. - Added the ability to edit File object content in a textarea. If the content of a File is a text type and the size is less than 64K (the max size for text area data in some Web browsers), then you can edit the content as you would a DTML or other text-based object. - New user management API for user folder objects was implemented. As of Zope 2.5, manage_addUser, manage_editUser and manage_delUsers form the official API for user management. The old grotesque way of using manage_users is now deprecated. The default implementation of these API methods simply call the _doXXX versions of the methods that user folder authors have already implemented, which means that these APIs will work for current user folder implementations without any action on the part of the author. User folder authors that implement the new manage_XXX API can get rid of the old _doXXX versions of the methods, which are no longer required (we only use them if the new api is not directly implemented). API documentation for the User Folder API was also added to the help system. - Python compiler in RestrictedPython brought in sync with main distribution. Made to be compatible with Python 2.2. - Added links in the "debugging information" control panel for viewing the contents of the ZODB cache. - Added user password encryption capability, fulfilling the needs described in the proposal at: http://dev.zope.org/Wikis/DevSite/Proposals/EncryptedUserfolderPasswords There is now a "properties" tab in user folders where you can select whether passwords are stored encrypted. - Locale support in STXNG has been broken (since 2.4.0) - Added SOFTWARE_HOME, INSTANCE_HOME, and CLIENT_HOME to the control panel to make it easer to tell what configuration of Zope is running. - Re-applied Medusa patch to allow proxy-like HTTP requests (GET http://hostname:port/ HTTP/1.0) - Objects indexed by the PathIndex can now provide a hook or an attribute with the same name as the index name to provide a customized path information other than using getPhysicalPath(). - Selecting "Clear Catalog" from the ZCatalog "Advanced" did not clear the vocabulary associated with a TextIndex. - Added updated and enhanced testsuite for STXNG. - added getEntryForObject() for PathIndexes. Reworked PathIndex's internal inverse index. - API help topics can now document functions as well as classes. - Security accelerations in c - Accelerated C Document Template handling; Document Templates can now do additional rendering in C, improving performance. - Unicode support for ZCatalog: - added new UnicodeSplitter - ZCatalog now allows unicode strings as attributes of objects to be indexed using a TextIndex (see lib/python/ZCatalog/README.txt) Bugs fixed - WebDAV: Zope escaped nested object properties derived from internal dav__* functions in PropertySheets.py although they are considered to be safe and do not need any escaping. This caused Zope to be completely incomplete with Windows XP. Fixed ! - WebDAV: '(' and ')' are now allowed in Ids for Zope objects. This is needed when one tries to duplicate files using cut&paste through Microsoft webfolders. - Collector #2532: ZCatalog.availableSplitters is now protected by security mechanism. - Collector #2412: a read-only FileStorage has not been closed properly. - Collector #2390: Objects of type 'Help Image' were not properly re-registered inside registerHelp(). - Fixed broken FTP download for larger files. - Collector # 2396: StructuredText did not allow URLs containing "%" - Collector # 2397: StructuredText could not handle underlined text properly. Also will no longer produce .. and .. - Collector #2438: Using a slice operation like [30:] on a ZCatalog search result caused a MemoryError because the __getslice__ implementation used range() instead of xrange(). - Collector #2423: Searching a FieldIndexes for documents with a blank string has been broken. - WebDAV Lockmanager was not working due to a Python 2.1 incompatibility. - Collector #2482: A COPY operation through WebDAV on a locked resource left the destination resource in a locked state so any WebDAV client was unable to unlock the destination object. Locks are now cleared from the destination object. - Error message AttributeError/_v_blocks when a DTMLfile is not present or could not be read replaced by a more informative message. - Collector #2497: SERVER_PROTOCOL variable is now compliant with the CGI specification and looks like "HTTP/1.1" instead of "1.1" - Creation of a TextIndex ignored the vocabulary setting. - Fixed broken aquisition of vocabularies from a Catalog by a TextIndex. - Collector #2504: level parameter has not been passed to HTMLClass constructor - default for 'orphan' attribute of is now 0 instead 3. - Fixed conflict resolution problem in BTrees (BTreeTemplace/ _p_resolveConflict) - Fixed many subtle bugs in the BTrees, including some that could silently corrupt the BTree and render some data inaccessible. - Collector #2524: Medusa sent "HTTP/None..." as response header when then HTTP version could not be determined from the HTTP request. Now sending "HTTP/1.0..." - queries for the PathIndex can now specified as tuple (path,level). the level parameter inside a query overrides the optional 'level' parameter for a complete search request. - Collector #2561: XXBucket.values() returned keys instead of values - Fixed the API docs for user objects. - Fixed bad interaction between ZCatalog and dtml-in (submitted by Steve Alexander) - Multiple links in a paragraph with mixed link notation (quotation+colon+URL or quotation+comma+whitespace+URL) did not work properly (thanks to Alastair Burt for reporting and submitting the patch). - Fixed case where DTMLMethod.py complained when it tried to remove the accelerated DTML security validation routine after recursive entry.