CreatingCustomStructuredText

With StructuredTextNG, you can create your own Custom StructuredText formats. For example, you can disable features you don't like, add features you miss, or change the way current features behave.

How does this work? Well, you can start from scratch and use your own custom parser, rather that the DOC parser included with StructuredTextNG. This is especially attractive if you want to create a structured text format that has nothing to do with documents.

If you want to start with the classic StructuredTextRules and make customizations, then you can do so by creating a custom parser based on the DOC parser by subclassing the DOC parser.

Work in progress ..... :)

How does the user define new types, extend/overload old one?

• To define a new type, the user would need to create a class for the new type. This class would contain the expr that matched the new type. The class __call__ method would be overloaded to receive a raw string and determine if it matched the new type. If the raw string matches, an new instance of the type is created and that instance's string becomes the matching sub-string. The instance also maintains the start and end positions of the sub-string in relation to the original string. The class also needs to maintain a span method, which returns a tuple (start,end), of the sub-string's position.

• To change how a type is matched the user would need to alter the expr in the class for the type to be changed.

• To ignore a type, the easy way is to remove the type from self.types in Doc. This class either be done brute force by literally removing it from the code, or by subclassing doc and simply splicing the type from self.type NOTE : this requires knowing the location of the type in the list

How to extend DOC to recognize new types

Define the new type

1. Need to write a new class for the type. This class must have the following

   • an overloaded call function

   • an overloaded init function

   • a span function

   • a type function

   • string function

2. The init function will create a self.str item for the new type. Also has two items for the span function, self.start, self.end

3. The type function will return string which tells what type the instance is. Ex : the current header class's .type() returns "header"

4. A string function which returns self.str

5. The overloaded call function receives a string and determines if there is a matching structured text type in the string. If there is, set self.start and self.end for the range of the sub-string that matches. Create a new instance whose string is the matching sub-string. Return the new instanace

6. span returns the tuple (self.start,self.end)

Make it so DOC can recognize the new type

1. Need to create a new DOC, which subclasses the old DOC

2. Overload the init function. Perform the original DOC init, but then self.types needs to be modified. This is a list of structured text types. An instance of the new type must be inserted/appended to the list. NOTE : Order does matter.

How to extend DOC to overload old types

Remove a type

• Sub-class ST.DOC, and overload the init function

• Splice the type to remove from self.types.

• Now use the sub-class of ST.DOC to accept the ST.StructuredText structure.

Modify an existing type

1. sub-class the existing type. Ex: class my_header(ST.header):

2. To modify what recognizes the structure, overload the init function. Perform the original init. There is a self.expr item which is how each type finds a matching sub-string. Modify this to fit your needs

3. If neccessary, modify the overloaded call function. It is up to the call function to receive a string and find any sub-strings in the string which match self.expr. If a sub-string is found, call will create an instance of the type, like the header call will create a header instance in this manner, result = header(sub-string). Call must also provide the result a start and end (result.start, result.end) which indicate the starting and ending positions of the substring in the original string. Call will return the newly created instance.

4. Sub-class ST.DOC and overload the init function. Call the original init, then in self.types, replace the previous class name with the new one. Ex: in the new DOC __init__ do. (For this example I will use header)

   • ST.DOC.__init__()

   • self.types[:2]? = my_header()

How to extend a parser

Sub-Class an older parser

1. Why Sub-class an older parser?

   • If the user is modifying a small number of structured text types, it is faster to sub-class and have the majority of the types pre-defined by an old parser.

   To add a new type

2. Modify the self.types for the parser class. This is a dictionary, so order is illrelevant. Ex: self.types["newtypename"]? = self.typefunction where newtypename is the string returned by the new instance's .type() call and typefunction is the function in the parser which handles that instance type.

3. Modify the self.self_par if the strucuture marks paragraphs internally. Ex: self.self_par.append("newtypename") where newtypename is the string returned by the new instance's .type() call. Headers and lists do this currently

   To overload a built-in type

4. In the new class, re-define the function which handles the type Ex: overloading header In the sub-class, re-define header def header(self,object): self.string = self.string + "I am not a header or crook"

5. Remember that functions receive instances of the types they handle. To go through the instance's string, use the .string() call.

6. The .string() call returns either a string (if the object's string is text only) or a list if the object's string contains other instances. If a list is returned it is necessary to go throught each item. There can be only three things in a list, strings, lists, and instances. For strings, call the self.paragraph function, or whatever function handles strings. For lists, call the self.loop function. For instances, call then self.instance function.