Finally! CKEditor integration with Lotus Domino

It's been over a year since I posted my demoapp of FCKEditor-integration with Lotus Domino.

The CKEditor (the new and improved!!! version of FCKEditor) has been out for a little while now. At first, the upload integration wasn't documented, so I didn't want to waste time reverse engineering the functionality.

I got a mail a couple of days ago from a reader about CKEditor-integration with Domino. I replied that I would create a demoapp of the integration as soon as the upload functionality was documented. Now it is (more or less).

The new API for FCKEditor is BIG, but that doesn't show on the CKEditor. Its fast as *insert something extremely fast*. Test out their demo if you don't believe me!

The only thing that's changed since the FCKEditor demoapp is the $$Return on the (f)ckupload form, and the hooks I use to rewrite the name of the upload field.

I tested the demoapp in IE8, Firefox 3.5, and Opera 10. Let me know if it doesn't work for you.

CKEditor must be installed in the ..data\domino\html\js folder (you have to create the js folder if you don't have one already). Test http://yourdomain.com/js/ckeditor/ckeditor.js to see if CKEditor is in the correct folder.

This also works on localhost, if you want to experiment on your own computer.

Read the FCKEditor-integration with Lotus Domino post as well. I may have forgot to mention some details.

Anyways, it's time to go to bed. Another day, another dime tomorrow.

>> Demoapp of integration with CKEditor

Serving correct MIME-types for Office 2007 attachments on the web

Several of our users had problems with opening Office 2007-attachments in Internet Explorer. The browser identified the attachments as zip files.

The workaround for this is to set up the Domino server to send the appropriate MIME-type header.

The correct headers

Then you need to configure the Domino Server. We hoped it would be as easy as adding config documents to the "File Identifications" view. Alas, this didn't work.

What did work was modifying the httpd file in the data-directory of the server/refreshing http
(tell http refresh).

#   Office 2007
AddType .docm      application/vnd.ms-word.document.macroEnabled.12
AddType .docx      application/vnd.openxmlformats-officedocument.wordprocessingml.document
AddType .dotm      application/vnd.ms-word.template.macroEnabled.12
AddType .dotx      application/vnd.openxmlformats-officedocument.wordprocessingml.template
AddType .potm      application/vnd.ms-powerpoint.template.macroEnabled.12
AddType .potx      application/vnd.openxmlformats-officedocument.presentationml.template
AddType .ppam      application/vnd.ms-powerpoint.addin.macroEnabled.12
AddType .ppsm      application/vnd.ms-powerpoint.slideshow.macroEnabled.12
AddType .ppsx      application/vnd.openxmlformats-officedocument.presentationml.slideshow
AddType .pptm      application/vnd.ms-powerpoint.presentation.macroEnabled.12
AddType .pptx      application/vnd.openxmlformats-officedocument.presentationml.presentation
AddType .xlam      application/vnd.ms-excel.addin.macroEnabled.12
AddType .xlsb      application/vnd.ms-excel.sheet.binary.macroEnabled.12
AddType .xlsm      application/vnd.ms-excel.sheet.macroEnabled.12
AddType .xlsx      application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
AddType .xltm      application/vnd.ms-excel.template.macroEnabled.12
AddType .xltx      application/vnd.openxmlformats-officedocument.spreadsheetml.template

Not sure if it is a bug that the "File Identifications" solution didn't work..? The server-version is Domino 8.5.

The HttpRequest class updated - Get cookie values

I've updated the HTTPRequest-class so that you also can use it to pick cookies of a request.

Example code:

Dim request As New HttpRequest()
Dim sessionCookie As String
sessionCookie = request.cookie( "DomAuthSessId" )
Print "I've got your cookie!"

Although I link to Jake's excellent demonstration of how simple it is to get the session cookie, I wrote the cookie-bit a couple of days before, in work relation.

>> HttpRequest Source code

Templating: Generating HTML from document collections

Sometimes it can be useful to render HTML from document collections. I believe one of the fastest ways to get values from Notes documents is using Evaluate. As a lazy person, I like to avoid writing formula code, as it's somewhat verbose compared to modern templating languages like PHP.

To avoid this, make your own template interpreter using simple formula code.

Example:

str := "Date: $#dayOfMonth#$. $?month?$.";
'"' + @ReplaceSubstring( str ; 
     "$?" : "?$" : "$#" : "#$" ;
     '" + ' : ' + "' : '" + @Text( ' : ') + "' 
) + '"'

Result: "Date: " + @Text( dayOfMonth) + ". " + month + "."

The result can be used in an evaluate statement.

A great thing about templating is that it is very maintainable/reusable. If you want to change the output, all you have to change is the template-string. To generate XML and HTML from the same collection, all you have to do is to have two different string templates. The conversion logic can be the same.

In the simple demoapp I've included, a collection of 300 documents is processed, extracting five fields. A HTML table row is generated per document. class="even"/class="odd" is added so that one can style odd/even rows. If you're somewhat familiar with LotusScript, it should not be hard to make a template-class to fit your needs.

On localhost (my laptop) it takes about 100 milliseconds to generate the HTML/write it to a PassTruHTML NotesRichTextItem.

>> Demoapp

If you want a sorted collection, you can use NotesView.AllEntries, and use NotesViewEntry.Document, to get to the document. Beware of categorized views. Test NotesViewEntry.isValid = True to be certain that the entry represents a NotesDocument. There may be other problems with using NotesViewEntry.Document as well. Proceed with caution/good error handling. :)

?OpenField - get HTML from a RichText-item

In case you missed it in Yellowcast, episode five. There's an undocumented feature in Lotus Domino that lets you pull HTML from a RichText field. This is a very useful feature for me at least.

Example url: http://www.notes.net/secrets/undocumented.nsf/documentation/easteregg?OpenField

I believe Carl Tyler was one of the first to blog about this.

Why not document such a great feature? Why, IBM?!

Json-search on Domino views

Domino (at least pre 8.5) doesn't let you use $$SearchTemplate forms as anything other than vanilla Domino-generated html forms.

This makes it somewhat difficult to search/getting Json-data in return.

The simplest workaround I've found is to make a view with one column. The data is formatted as Json, with a trailing comma. In the search-template, I wrap the $$ViewBody-field like this:
<div id="result">[|$$ViewBody-field|,""]</div>

When I want to search, I use a temporary iframe, added to the DOM using Javascript. To make it as simple as possible, browser compatibility-wise (events), I add a the iframe inside a temporary div using innerHTML. The iframe has an onload-event. When that triggers, I know that the Json is loaded.

I'm terrible at explaining code (terrible at explaining anything), so I've made a simple demo-app you can try. Remember to create a full-text index on the demo-db before trying it out.

Flash demonstration of the app (I parse the Json to HTML):




>> Demoapp

The app is tested, and works in Opera 9.5, FF3 and IE7 on Vista x64.

Slick integration between FCKeditor and Lotus Domino

Update 16.08.08: According to a commenter, Sinisa, the notes.ini modification is not needed (tested on Domino R7), so even less requirements.. :)

Before you read on. The integration requires you to set DominoDisableFileUploadChecks=1 in notes.ini on the Domino Server.

I found no satisfying integration-examples that included upload-support for FCKeditor, so I decided to try to make the integration myself. As far as I've tested, you can upload whatever you want, only limited by settings in FCK and Domino restrictions.

Flash animation of the integration in action. Shows upload of an image and upload of a flash game.

Although it was a bit hard to hack around Domino restrictions regarding uploading files, I finally found out how to make it work.

All that was needed was a bit of try, fail, retry, repeat, a simple upload-form and a little bit of Javascript.

Demo application

>> Download

The interesting bits are the fckconfig.js page, the demo form, and the fckupload form. Read Ferdy's article to understand how you can upload files to Domino without using a Domino generated File Upload Control.

Requirements for the demoapp:
fckeditor in [Notes/Domino-path]\data\domino\html\js\fckeditor
If you're not running the demo on localhost, DominoDisableFileUploadChecks=1 in notes.ini on the server.

Tested with FCKeditor 2.6.1 in FF2.0, IE7(Vista) and Opera 9.5 (go try it out).

A couple of handy cheat sheets

I use both of these on a regular basis. I've printed them out on paper, so they're always withing arms reach.

Domino URL cheat sheet

Domino supported syntax for Full Text searching

For web development, ILoveJackDaniels.com has a few nice ones (the site seems to be down at the moment).

Setting charset in pages/forms with content-type: text/javascript

The Character set-setting for a page/form doesn't seem to do anything more than to set the character encoding for the text if content-type is text/javascript.

With character set set to Unicode (UTF-8), and content-type to text/javascript, Norwegian letters look like this:


If I set content-type to: text/javascript; charset=utf-8



Much better.. :)

A small simplification to "lookups" on the web

In this post, I referred to a method of looking up one document in a view, using the parameters startkey and untilkey.

E.g.

http://server/db.nsf/view?openview
&startkey=alex
&untilkey=alex_



I discovered today (wonder why it took so long) that the good old count-parameter works just as well as the untilkey-parameter. Personally, I think that the count-parameter is more readable than the previous method.

Simplified url:

http://server/db.nsf/view?openview
&startkey=alex
&count=1



I don't know if there is a performance-difference between the two methods. I tried a couple of benchmarks in firebug, but got nothing conclusive.

Improved HttpRequest class - Handles request_content above 64k

For those of you that haven't heard, in Domino 7.01, IBM implemented a workaround for (HTTP POST) request-content above 64k.

I've updated the HttpRequest class, so that it handles requests above 64k automagically.

>> My testbed for the class
The test-app has a page that posts ~260k of data to an agent. This results in 4 request_content fields. The agent prints a report of the request ( [HtppRequest].printHtmlReport ).

The class (txt-file)

Let me know if there are errors.

Embed multiple standalone forms on a domino form using DHTML

By standalone I mean that when you submit a form, you don't submit all the others/the main form.

I decided to make this when I read a question in Jake's latest article, How to Embed a Login Form on All Lotus Domino Pages.

The question is not directly connected to this blogpost, but somewhat.

The question:
Hi Jake,

We are getting two HTML form tag in view source of page in browser, one form tag for main form and other which is login form.

My question is, How will I handle when we are submiiting the Login form ? Will it submit the main form as well ?

Could you please help me

Regards
Ajay B Mali


The demoapp loads another form using an iframe. Using DHTML I pick the form out of the iframe and inject it at the top or bottom of the form that is open. The reason for either top or bottom is that forms can't be nested (read Jake's excellent article for more info). If you want it in a specific position on the page, CSS may help you.

The reason for iframe and not XHR (Ajax), is that XHR returns text. With the iframe, I can more or less pick what I want out of the "iframed" document, the way I would on the "main" document.

>> The demoapp (Names-form)

Tested in Opera 9, IE7 (Vista) and Firefox 2

Serving Gzipped content - Test for browser-support

In the newest version of the CMS I'm working on/maintaining, a couple of customers got strange errors. The culprit was that for some reason or other, their browser didn't accept gzipped content.

The workaround

acceptsGzip := ( @Contains( @GetHTTPHeader( "Accept-Encoding" ) ; "gzip" ) );

{<script src="/} + @WebDbName +
@If( acceptsGzip ; "/gz/mootools.js" ; "/lib/mootools.js" ) + {"></script>}

Taking visual control of Domino generated views

This is somewhat related to my previous post, as that is what sprung this idea forward, adding a class-attribute per even row/colgroup-nodes per column, dynamically.

In the downloadable demo application, I've made a $$ViewTemplateDefault-form with a JS-function that does the above, onload.

Every even row has class="even". The generated colgroup is structured like this, a <colgroup> containing <col id="column[number]"/> nodes, where [number] corresponds to the column number it represents.

To get a fix on the Domino-generated table, I've added a span around an embedded view (to easier control count). id="domino_view", class="@ViewTitle[1]".

I don't know what more to say than take a look at the screenshots, and download the app, look at the code/css if you think it's something you can use.

Some screenshots

Categorized view, no CSS
Categorized view, styled

Flat view, no CSS
Flat view, styled

>> The demo application

Tested in IE 6/7, FF2 and Opera 9.

If you discover bugs/find something strange in the demo-app/etc, let me know. Preferably through comments.. :)

Readable lookups - Demo application

The demo contains:

• Implementations of the readable lookups-technique


• Examples of my "Semi-MVC" templating technique.


• A demo of in-field transformation with the formula language.


• A demo of Ajax-lookups using the "readable lookups"-view I use in the formula-demo, and a simple templating/transformation demo using DHTML.

I've made a simple index-page. If you download the app/open http://domain.com/path/to/demo.nsf, you should see the links to the demos.

The demos aren't very exciting/useable. Take a look at the design elements for the "magic".

The Ajax-demo

Link "To the frontpage" is a part of the above flash-animation

>> Download demo-application

Doing lookups on the web

Update, 20.12.07: Demo-application done.

I've finally gotten started with my templating-demo again.

One of the examples I promised was Ajaxy lookups (using the same view on the Web as in Notes). To make this happen, I thought it would be enough going to use ?OpenView&startkey=hello, but the effect of that "command" is start with documents starting with hello, then show the rest of the documents in the view.

A quick visit to my friend, google, sendt me along to an article on CodeStore.net. "Hidden" in the comments-section was my answer:

Alex Hernandez (Thu 5 May 2005 10:00) website / e-mail

Jake, I've been facing a similar problem when reading the "startkey", if the key doesn't exist, it returns the next set of documents, which is wrong. Now, if you add the "untilkey" to the url you can grab only the documents you need. This is true everytime you know how to compose the untilkey key, What I did was just adding a "_" to the end of my know startkey.

for example:
http://server/db.nsf/view?openview
&startkey=alex
&untilkey=alex_

originally If I searched for "alexh" and the key doesn't exist, it returned the rest of the docs. now with the until key, it search for docs with ket between "alexh" and "alexh_" wich returns no documents, and that's what I expected.

.::AleX::.

Thank you Alex, Jake and google!

Templating using a NotesForm

Coding templates in an agent leads to very verbose agents/a maintenance nightmare (in my opinion). Coding them in a NotesDocument makes it hard to use them in a Design-template context.

This technique has little verbosity beyond the actual code to generate HTML, and works great in a Design Template-context. It supports modularity, as each field is a potential module. Since the templates are forms, the dynamic "modules" also work in stored documents.

You can combine Java (through LS2J), formula (through Evaluate) and LS , just like you can when you're using agents for templating web-pages. If you're a Java-guy, then you probably would need a little more code (testing field names with Regular Expression) to achieve the same.

Combine this technique with @UrlQueryString in Form Formula if you want the ability to present data in several content-types.

The technique relies on Execute, a WQO-agent and field-names following a pattern. I chose Execute because it's faster than making a testing-routine for patterns. If you're planning on putting this technique to use in real applications, I suggest you use Regular Expressions, Instr, etc. to look for valid field-names, as Execute is horrible to debug.

The demo

I have one pattern that I look for in a field-name, "wqo_actonfield_". The third token (separator = "_") is the method to be executed. The rest of the tokens are ignored to allow several fields executing the same function.

You could also have the rest of the tokens being parameters for the method being executed.

For instance field named "wqo_search_google_dontpanic" -> run a function that inserts HTML with search-results into the field.

>> Demo Application.

Screenshots

WQO-agent for demo:


Form, one demoBody-field:


Result, web


Form, six demoBody-fields:


Result, web (click for full size)



As always, comments are appreciated :)

NotesSession.SavedData - great for WQO-agents?

Update 3:
I got a little more useful information from Fabian today:
For whatever reason, even with the $PublicAccess item set to "1", Anonymous still needs at least reader or depositor access to actually write to the agent data note. I usually prefer to restrict public readers to No access with the privilege to read and write public documents, whenever possible (there is another oddity with shared fields, who don't have a GUI widget to enable public access). Reading item values works OK that way, but writing not.

Obviously (at least in certain releases) Domino doesn't do a perfect job deleting agent data notes when agents are deleted. IBM has a little tool to purge orphaned agent data notes, but TeamStudio has a better (and free) utility.


Update 2:
I got this by e-mail from a friendly German, named Fabian Brock.

If the WQO agent is not run as web user, there should be no problem whatsoever. If the signer of the agent has appropriate rights to the database, he/she can modify the agent data note just like any other document. However, anonymous users will generally not have the right to create or even edit documents.

The solution is the same as with conventional docs: Have your WQO agent add a $PublicAccess item set to "1", if it doesn't exist yet. Now, Anonymous is only required to have NoAccess plus the privileges to read and create public documents. Major drawback here, since SavedData is not available for public access yet, Anonymous will never be able to add this item. The agent must have been run once by a user with sufficient access rights to make the modification.

And because the agent data note is recreated every time you make a modification to your agent, this might turn out to be hard to handle. One more reason to put as much of your code as possible into libraries.


Update:
It seems (from a little testing) SavedData is not available for web-agents, which is a shame. I tested this in both Java and LotusScript, and got the same results. Next best solution, use a profile-document.

>> Demo-code (java agent), profile document

--

When I first read through Julians benchmark of different ways of String concatenation, I only skimmed through to the nice graph.

I read it again, and saw that he mentioned NotesSession.SavedData which I never heard of. Looked it up in the help, and got an idea (good? you decide). Wouldn't this be a great container for HTML created by WQO?

WQO that prints data from documents (e.g. menus/reports):
Create a lookup-view with a column containing @Modified.

When you run the agent,
Join(Evaluate(@Text(@DbColumn( "" : "nocache" ; ... )))) -> a string of all the dates. Compare this to a stored field in SavedData containing the @Modified from the previous run.

If the same/SavedData has "date-field", replace RT-field in DocumentContext with the saved field in SavedData, else generate HTML/store in SavedData.

Click-to-sort "view" in one column

In this SNTT, I try to demonstrate some of the power of @Sort and Evaluate.

The demo I've made transforms a column containing pipe-separated values to a HTML table with click-to sort (Ascending/Descending) column headers, using a form with a WebQueryOpen-agent.

>> Flash demo

In the flash-demo, performance may seem a little low. The demo is running locally. I tested the form+agent in another application on a Domino-server, to see if it was as general as I wanted, and there, performance was a lot better. It took me about a minute or two to make a click-to-sort view in the other application, which I'm quite happy about.

Domino has click-to-sort columns in views, but I've never tested it on the web. The strength of this way of doing it is that you have the power over the HTML.

There are 20k+ documents in the demo-application, but the agent that generates the click-to-sort tables can only handle 100-1000 rows of data. I couldn't find a corrolation between the amount of data, and max number of rows. I tested a two-column table and a four column table, and I got more rows out of the four column one.

At the top of the MakeSortableTable-agent, there is a parameter (maxrows) that controls the max amount of rows to show in the view.

--

>> Demo Application

If you want to add this functionality to an application of yours, simply copy the form and agent in the demo into your application, make the "lookupview", in the same way I made my People-view. Column title should contain the table header titles (separated by "|") for the columns, column value should contain the column values for the table (also separated by "|")

Column1 in the People-view


Column2

Rewriting the renderer to Java

Continuing my templating-experiment in Domino.

I'm doing this for two reasons, to brush up on my Java-skills (or lack thereof), and to make the templating syntax simpler (using the power of Regular Expressions).

Evaluating formulas are possible.

LotusScripts blocks not possible directly. You could write LS-statements to a field in a document, run a LS-agent that Executes the string, and prints the result in the same field. Open the document again in Java, and fill in the result, but I think that would be very hard to debug.

I hope to use this syntax (please give input if and why this is bad):
%moduleName% <- a module
$fieldName <- reference a field, wherever in the template
<@ .. @> <- formula block

Formula-blocks makes it possible to do stuff like:
<html>
<head><title>UFO Sightings</title></head>
<body>
<@

@If( @UserRoles = "[Editor]" ; "" ; @Return("<h1 style=\"color:red\">Access denied</h1>") );

lup := @DbLookup( "" ; "" ; "(lupTopSecret)" ; "ufo" );

"<h1>UFO Sightings</h1><ul>" +
@Implode( "<li>" + lup + "</li>" ; "" ) + "</ul>"

@>
</body>
</html>

This would be a standalone web-page, not a template referencing data from a stored document.

The above example is of course possible to do with a Form or a Page.

I'll see how long my interest in this experiment last. Hopefully I can make a demo that is hard to do with existing technology.

For instance,
..
<div id="nav">%menu%</div>
<div id="content">
<xmlTransform src="http://webpage.com/view?ReadViewEntries"

xslt="http://webpage.com/fancyTable.xsl" />

</div>
..