Generating HTML on WQO with Formula-agents

If you mark a RT-field Pass-Thru HTML (add a space before and after, Text -> Pass-Thru HTML), you can fill it with HTML through formula.



You are limited to ~64k of data per field, and you will have to experiment a little with the generation of the HTML-string (using an array/implode), as Domino tends to give this error-message: HTTP Web Server: Lotus Notes Exception - The formula has exceeded the maximum allowable memory usage.

The advantage with WQO compared to setting the HTML in "Default value"/using a computed text is that you can move the code out of the form (I prefer as little as possible code in the form). You can run multiple agents on WQO if you have several RT-fields you want to fill (menu, maincontent, etc).

When generating HTML, I've been using more and more Evaluates inside LS-WQO-agents. In some cases, where the generated html is less than 64k, formula-agents may be the best "tool".

>> Demo-db with different formula-agents running by @UrlQueryString/a form with two formula WQO-agents.

onBeforeUnload - confirm moving away from page

After reading Patricks post about onbeforeunload, I implemented the event in the CMS I mostly work with at Compendia.

His code required Prototype. I've made a little demo that doesn't require a JS-lib. I use two event-listeners ( onClick/onKeypress ) on document, and "set the page dirty" when the event "indicates" that a form element has gotten its value changed (look at the sourcecode of the page).

The only weakness I've found so far is that IE6 doesn't register keypress on comboboxes -> user could change a combobox-value with the keyboard and move away from the form unsaved without getting a warning.

>> Demo-page

I think onBeforeUnload is supported i FF2.x, IE 4+, and newer versions of Safari.

Mozilla has a simpler demo, where one field is tested.

Patrick has a downloadable Demo-NSF.

Keyword/Value lookup fields

I sometimes get a bit frustrated by the lack of associative arrays in the formula language (more specifically in keyword lookup-fields). I’ve thought about ways to emulate this a couple of times, but never got anywhere.

Today, I’ve come up with one way. It’s not beautiful, and one can argue that it’s bad practice.

lupField-syntax:
standard @DbLookup( .. )

Column Value-syntax:
"Error message - keyword-list not found" :
"keyword" : value
"apple" : banana;

Syntax for getting value:
lupField[ @Member( "keyword" ; lupField ) + 1 ]

Error handling

• Error message on "keyword value not found" maintained on a column basis.
• First item in the Column Value list should be the error message. Reason being @Member( "itemNotInList" ) = 0 (+1=1)
  
• "keyword-list not found" maintained in the lookup-field.
  

"Advantages":

• Put keyword/value wherever you want in the column formula
• Great for design-templates if you want to have configuration-settings in a logical order in the lookup-column (reorder them whenever)
  
• Print all settings through a simple @For-loop
• Readable
• @Word( settings ; "|" ; 24 )
• versus
  
• settings[ @Member( "stylesheet" ; settings) + 1 ]
• ..?
  

Disadvantages:

• Unreadable syntax for people not aware of the technique
• ..?
  

>> Ugly flash-demo
>> Ugly demo-application

If any of you have an even simpler syntax/better concept, please let me know!

String to char-array in Java

For those curious Java versus LS. It's lightning fast:

String to char[]: 15ms, string length, 700 000

Added to the string concatenation "benchmark"

String concatenation with a Java agent

Here are the results from string concatenation in Java:
Standard concatenation (75 000 concatenations): 192.594s, string length, 300 000.

java.lang.StringBuffer-concatenation (75 000 concatenations): 0.062s, string length, 300 000.

RTI-concatenation (75000 concatenations): 1.765s, string length, 300 000.

The best in LS, 75 000 concatenations, 0.25s. The numbers aren't directly comparable, as I'm running on different hardware than Julian.

By "Standard" concatenation, I mean bigString += string. Standard concatenation and RT-concatenation is slower in Java than in LS. Using Java's native StringBuffer class, is a lot faster than any of the competitors in LS (so far).

I'm not surprised that Java's StringBuffer beats Julian's array-based StringBuffer-class in LS, as Sun probably puts a lot more effort into optimization of the language than IBM does with LS.

My test ran on Notes 7.02, which I think runs version 1.4 of the Java runtime. Newer versions of Java may be even faster.

My Java-skills are mediocre at best. Let me know if my test-methodology is wrong in any way.

>> Code for the test

NotesSession.SavedData - great for WQO-agents?

Update 3:
I got a little more useful information from Fabian today:
For whatever reason, even with the $PublicAccess item set to "1", Anonymous still needs at least reader or depositor access to actually write to the agent data note. I usually prefer to restrict public readers to No access with the privilege to read and write public documents, whenever possible (there is another oddity with shared fields, who don't have a GUI widget to enable public access). Reading item values works OK that way, but writing not.

Obviously (at least in certain releases) Domino doesn't do a perfect job deleting agent data notes when agents are deleted. IBM has a little tool to purge orphaned agent data notes, but TeamStudio has a better (and free) utility.


Update 2:
I got this by e-mail from a friendly German, named Fabian Brock.

If the WQO agent is not run as web user, there should be no problem whatsoever. If the signer of the agent has appropriate rights to the database, he/she can modify the agent data note just like any other document. However, anonymous users will generally not have the right to create or even edit documents.

The solution is the same as with conventional docs: Have your WQO agent add a $PublicAccess item set to "1", if it doesn't exist yet. Now, Anonymous is only required to have NoAccess plus the privileges to read and create public documents. Major drawback here, since SavedData is not available for public access yet, Anonymous will never be able to add this item. The agent must have been run once by a user with sufficient access rights to make the modification.

And because the agent data note is recreated every time you make a modification to your agent, this might turn out to be hard to handle. One more reason to put as much of your code as possible into libraries.


Update:
It seems (from a little testing) SavedData is not available for web-agents, which is a shame. I tested this in both Java and LotusScript, and got the same results. Next best solution, use a profile-document.

>> Demo-code (java agent), profile document

--

When I first read through Julians benchmark of different ways of String concatenation, I only skimmed through to the nice graph.

I read it again, and saw that he mentioned NotesSession.SavedData which I never heard of. Looked it up in the help, and got an idea (good? you decide). Wouldn't this be a great container for HTML created by WQO?

WQO that prints data from documents (e.g. menus/reports):
Create a lookup-view with a column containing @Modified.

When you run the agent,
Join(Evaluate(@Text(@DbColumn( "" : "nocache" ; ... )))) -> a string of all the dates. Compare this to a stored field in SavedData containing the @Modified from the previous run.

If the same/SavedData has "date-field", replace RT-field in DocumentContext with the saved field in SavedData, else generate HTML/store in SavedData.

Redim performance

Out of curiosity/"challenge" from Julian, I did a little test of ReDim performance.

I'm not that familiar with redimming arrays, so someone let me know if this way of testing is bad.

Result:

Code:


Percentwise, the difference is humongous, but in seconds, not that big of a difference.

>> Code as text

Simple header-generator in Java

Not sure if header-generator is the best word for it.

Fonts aren't platform independent. An image of a font is.

Just a little experiment with Graphics2D in Java. JavaAgent running on WebQuerySave generates an image (and embeds it in the document) from header-text, selected font/-size. Also adds a shadow if desirable.

Width is currently hard-coded (400px), so the biggest fonts, with font-size set to 96 won't fit.

Fonts are @FontList. Since this is not available on web, I made a keyword-document containing the text-list of fonts. Not all types of fonts are supported, it seems.

I use a Java WQO-agent that sets a multivalue field containing all font names in GraphicsEnvironment.getLocalGraphicsEnvironment().getAllFonts(). This way, you should get the fonts available on your machine/server.

Header-images are saved as png in the systems tempfolder.

>> DemoDB

Examples:

LotusScript and Java agents on WQO

If you ever were wondering, yes, you can run both a JavaAgent and a NotesAgent on WQO. Write to the same RichTextItem/etc.

WQO in form:


The LS-agent:


The Java-agent:

(body.update() not needed.)

Result:

Thinking outside the box, String Concatenation

If you're looking for the ultimate concatenation-tool:
Using Julians StringBuffer: 0.25s
Using NotesStream: 0.6s
Using NotesRichTextItem: 1.6s

==

A little while ago, I posted that NotesRichTextItem.AppendText is FAST.

On the train today home from work, I got an idea... Why not use a temporary NotesRichTextItem for string concatenation.

With 100.000 string concatenations, string = string + fourLetterString took about 140 seconds.

With NRTI.AT, it was finished in 1.6 seconds.. Read that again.. 100.000 concatenations in 1.6 seconds!

Simply do NRTI.GetUnformattedText to extract the concatenated string after you're done.

Code:

Result:


>> Download code

Ballmer Peak

Got this from a colleague today..

Scientific proof that programming and beer mix well.. :)

@Transform as Forall-loop

I didn't think this fit into the "Complex transforms".

To use @Transform as Forall loop, simply add a @Do inside @Transform, do whatever you want with the items, and have a return-value for @Transform (@Nothing,"", etc).

Example:

list := "Tommy|Valand" : "Ford|Prefect";

@Transform( list ; "item" ; @Do(

html := html : "<tr><td>" :
@Word( item ; "|" ; 1 ) : "</td><td>" :
@Word( item ; "|" ; 2 ) : "</td></tr>";

item
)

);
@Implode( "<table>" : html : "</table>" ; "" )


The above example isn't good, as using @For actually results in less code:

list := "Tommy|Valand" : "Ford|Prefect";

@For( i := 1 ; i <= @Elements(list) ; i := i + 1;

html := html : "<tr><td>" :
@Word( list[i] ; "|" ; 1 ) : "</td><td>" :
@Word( list[i] ; "|" ; 2 ) : "</td></tr>";

);
@Implode( "<table>" : html : "</table>" ; "" )


When concatenating large strings, you'll get better performance connecting the "snippets" in a list ( list := list : item ), than using regular string-concatenation ( string := string + item )

Complex transforms

I think there is a assumption amongst Notes/Domino developers that you can only have @If inside @Transform (I blame the documentation). This is wrong.

If you put a @Do inside the @Transform, you can have "as many lines as you want". Variables created inside the @Transform is reachable from outside the transform.

To document this, I've made an ugly example :)

Result:


Code:


>> Download formula

One Liners in LotusScript

Update, 15.10.07
Discovered that Run/RunOnServer also is "chainable"

Call s.CurrentDatabase.GetAgent( "TestWQO1" ).Run()
Call s.CurrentDatabase.GetAgent( "TestWQO1" ).RunOnServer()

Call s.GetDatabase( "server", "path" ).GetAgent( "TestWQO1" ).Run()
Call s.GetDatabase( "server", "path" ).GetAgent( "TestWQO1" ).RunOnServer()

==

Actually two lines, since you have to have a NotesSession-variable.
Dim s As New NotesSession

Get document:
s.CurrentDatabase.GetView( "viewname" )._
GetDocumentByKey( "key" )

You'd normally want to assign this to a variable -> more lines..

Based on the above, I thought you could do:
s.GetDatabase( "server", "path" ).GetView( "viewname" )._
GetDocumentByKey( "key" )

But it seems that a NotesDocument needs it parent database in memory. CurrentDatabase is a property of NotesSession.

Empty a view:
s.CurrentDatabase.GetView( "viewname" )._
AllEntries.RemoveAll( True )

s.GetDatabase( "server", "path" ).GetView( "viewname" )._
AllEntries.RemoveAll( True )

Empty a db:
s.CurrentDatabase.AllDocuments.RemoveAll( True )
s.GetDatabase( "server", "path" ).RemoveAll( True )

Delete documents based on form/etc:
s.CurrentDatabase.Search( |Form="SomeForm"| )._
RemoveAll( True )

s.GetDatabase( "server", "path" )._
Search( |Form="SomeForm"| ).RemoveAll( True )

AllDocuments and Search returns NotesDocumentCollection, so you could also use StampAll, etc.

@UrlQueryString in Form Formula

I briefly mentioned in my "PHP-like templating in Domino" post, but I think it deserves it's own little tip-post.

You can use @UrlQueryString in form-formula of a view, to decide which form to open a document in.

I may remember wrong, but I think you can do comparison like @UrlQueryString = "print" for "stand-alone" parameters, since @UrlQueryString without a parameter returns a text-list of all request-parameters.

Example of form formula in view, "myview":
@If(
@UrlQueryString = "print" ; "print";
@UrlQueryString = "json" ; "json" ;
@UrlQueryString = "xml" ; "xml" ;
form
)

http://mysite.com/index.nsf/myview/blogpost?OpenDocument
^document is opened in default form

http://mysite.com/index.nsf/myview/blogpost?OpenDocument&print
^form that is printer-friendly

http://mysite.com/index.nsf/myview/blogpost?OpenDocument&json
^form that makes json of data in document

http://mysite.com/index.nsf/myview/blogpost?OpenDocument&xml
^form that makes xml of data in document

These are not Domino-parameters. You have to make the forms yourself.. :)

Click-to-sort "view" in one column

In this SNTT, I try to demonstrate some of the power of @Sort and Evaluate.

The demo I've made transforms a column containing pipe-separated values to a HTML table with click-to sort (Ascending/Descending) column headers, using a form with a WebQueryOpen-agent.

>> Flash demo

In the flash-demo, performance may seem a little low. The demo is running locally. I tested the form+agent in another application on a Domino-server, to see if it was as general as I wanted, and there, performance was a lot better. It took me about a minute or two to make a click-to-sort view in the other application, which I'm quite happy about.

Domino has click-to-sort columns in views, but I've never tested it on the web. The strength of this way of doing it is that you have the power over the HTML.

There are 20k+ documents in the demo-application, but the agent that generates the click-to-sort tables can only handle 100-1000 rows of data. I couldn't find a corrolation between the amount of data, and max number of rows. I tested a two-column table and a four column table, and I got more rows out of the four column one.

At the top of the MakeSortableTable-agent, there is a parameter (maxrows) that controls the max amount of rows to show in the view.

--

>> Demo Application

If you want to add this functionality to an application of yours, simply copy the form and agent in the demo into your application, make the "lookupview", in the same way I made my People-view. Column title should contain the table header titles (separated by "|") for the columns, column value should contain the column values for the table (also separated by "|")

Column1 in the People-view


Column2

NotesRichTextItem.AppendText is FAST

Did a little benchmark today.

Code:

Dim startTime As Single, i As Integer,_
 streng As String
 startTime = Timer() 
 
 For i = 1 To 20000
  streng = streng + "1234"  
 Next
 
 Call htmlbod.write( streng )
 Call htmlbod.write( |<h1>Concatenation: | +_
 Cstr( Round(Timer() - startTime, 4) ) + |</h1>| )
 
 startTime = Timer()
 
 For i = 1 To 20000  
  Call htmlbod.write( "1234" )
 Next
 
 Call htmlbod.write( |<h1>AppendText: | +_
 Cstr( Round(Timer() - startTime, 4) ) + |</h1>| )

Result
Concatenation: 1,7188
AppendText: ,1719

The bigger the string-pieces, the more difference. Not a real-life benchmark, but at least it indicates that AppendText is lightning fast.

Simple LS2J Regular Expression class

A simple Regular Expression class. You may need to remove the line-breaks I've inserted to make it fit the width of the blog.

/*
Usage: 
 Uselsx "*javacon"
 Use "whateverYouCallThisLS2J-class"
 --
 Dim js As JAVASESSION
 Dim regexp As JAVACLASS
 
 Set js =  New JAVASESSION
 Set regexp = js.GetClass("RegExp")
 
 Print regexp.test("http://www.compendia.no",
  ".*compendia.*") -> True
 Print regexp.split("test","e") -> "t,st"
 Print regexp.replace("test pest hest", 
  "t\\w"," ", true) -> "tes pes hes"
*/

public class RegExp {
 //test a string
 public static boolean test(String streng, String expr){  
  return streng.matches(expr);
 }
 //returns a comma separated string
 public static String split(String streng, String expr){
  String temp[] = streng.split(expr);
  if(temp.length == 0) return "";
  
  String res = temp[0];
  for(int i=1; i < temp.length; i++){
   if(temp[i].length()==0) continue;
   res += "," + temp[i];
  }
  return res;
 }
 //replace parts of a string
 public static String replace(String streng, 
   String from, String to, boolean all){
  return all ? 
   streng.replaceAll(from, to) :
   streng.replaceFirst(from, to);   
 }
  
}

Rewriting the renderer to Java

Continuing my templating-experiment in Domino.

I'm doing this for two reasons, to brush up on my Java-skills (or lack thereof), and to make the templating syntax simpler (using the power of Regular Expressions).

Evaluating formulas are possible.

LotusScripts blocks not possible directly. You could write LS-statements to a field in a document, run a LS-agent that Executes the string, and prints the result in the same field. Open the document again in Java, and fill in the result, but I think that would be very hard to debug.

I hope to use this syntax (please give input if and why this is bad):
%moduleName% <- a module
$fieldName <- reference a field, wherever in the template
<@ .. @> <- formula block

Formula-blocks makes it possible to do stuff like:
<html>
<head><title>UFO Sightings</title></head>
<body>
<@

@If( @UserRoles = "[Editor]" ; "" ; @Return("<h1 style=\"color:red\">Access denied</h1>") );

lup := @DbLookup( "" ; "" ; "(lupTopSecret)" ; "ufo" );

"<h1>UFO Sightings</h1><ul>" +
@Implode( "<li>" + lup + "</li>" ; "" ) + "</ul>"

@>
</body>
</html>

This would be a standalone web-page, not a template referencing data from a stored document.

The above example is of course possible to do with a Form or a Page.

I'll see how long my interest in this experiment last. Hopefully I can make a demo that is hard to do with existing technology.

For instance,
..
<div id="nav">%menu%</div>
<div id="content">
<xmlTransform src="http://webpage.com/view?ReadViewEntries"

xslt="http://webpage.com/fancyTable.xsl" />

</div>
..

Simplifying Evaluate

If you're only interested in getting a single value from Evaluate, you don't need to Dim a Variant to containt the Array Evaluate (almost) always returns.

Simply implode the evaluate, and you can assign the Evaluate directly.

Dim commonUsername As String
commonUsername = Implode( Evaluate( |@Name( [CN] ; @UserName )| ) )