Solprovider Thoughts Kongregate Registration for Free Flash Games and Chat with solprovider

Lotus Notes, DB2, and J2EE

Originally written 20020612

Lotus has announced that the future of Lotus Notes is integration with DB2 and J2EE. I believe this will lose the audience that uses and pushes Lotus Notes.

DB2 is a great relational database platform. It does its job well. Lotus Notes is a document based database platform. It is the only major commercial platform of its kind. It does its job well, and it matches the real world better than any platform that has been built so far in history. J2EE is a more complicated version of Java solving ease of duplicating applications on multiple servers. Lotus Notes allows the easiest duplication of applications on a single server and multiple servers of any technology in existence. The templating system allows easy updates of all applications on all servers. Lotus Notes solves issues that J2EE cannot.


The challenge of every software developer is to build a useful model for data. The challenge is usually because computers like to work with multiples of items that are identical. If every item has the same length, then we can get record x by finding position (x - 1) times record length. [Example for you non-techies, if the record length is 10, then the first record is at 0 * 10 = 0, the second record is at 1 * 10 = 10, the third is at 20, and so on.] Computers like to work with databases with records of the same length because if they need record 4578, they can find it without looking at the data.

These databases are called "relational", because you rarely know the record number. If you want to find the employee named "John", you could check every record for the name "John". But programmers found that if you created a second "table" containing the names and the record numbers, and sorted by name, you could use a fast search algorithm to find the record. [A "table" is a chart. Each column is "field" containing a piece of information. A "table is usually sorted by a particular "field".] So instead of looking through all the records, we look at the second table, quickly find "John", get the record number of the data in the frst table from the record in the second table, and then quickly get the record in the first table. Because the second table contains information about the information in the first table, we say they are "related", and the database that contains both tables is "relational".

Relational databases are very fast, because the records in a particular table are always the same size. While this is extremely good from a performance perspective, it has basically caused the computer industry to employ people who are good at reducing real world data to fixed size records. A good computer programmer knows to use 20 letters for first names, and 30 letters for last names. In the early days, storage space was costly, so they would use 8 letters for the first name and 12 letters for the last name. While this caused complaints from people whose names were truncated, saving 30 letters each from thousands of records kept the costs low.

To have "one-to-many" relationships, programmers would add a third table. To store the names and ages of the children of an employee, a third table containing their information and the record number of their parent would be created. So if we want to find John's children, we find John in the second table, get his record number, and then get all of the records from the third table that have his record number.

We keep adding tables until the database stores enough information to make our application useful enough so the company will keep employing the programmer.

Sound confusing? Sound like there was lots of wasted space? Many male American first names have around 6 letters. Since the first name field must be large enough to hold an acceptable amount of the largest name, there were always many records that contained extra spaces, and someone would still be complaining about having their name truncated. (And try not to think about street addresses.)

The creators of Lotus Notes found a different approach. They built a document-based database system. Each record was a document that had certain characteristics that let it work with the system. Then you could add any amount of fields of any length. And some of the fields could contain multiple pieces of data.

So Bob has a document:
FirstName="Bob", Address="5 Oak St.", Hobby="sailing".
Jonathan has a document:
FirstName="Jonathan", Address="15 Wanamassa Point Road", Children="Michael,10";"Jennifer,8","Samantha,5","Elena,2".
This system allows us to add information at any time. If these are customers, our sales force can discuss sailing with Bob and the joys of a large family with Jonathan. And no storage space is wasted.

The real advantage of the new system was not saving space. It was because the programmer who could reduce the world to columns was no longer needed. The new developer was someone who could build an interface to allow the information to be entered easily. The people who programmed Lotus Notes had just left jobs as managers and secretaries. They learned how to build applications related to the real world. If it was decided to add a piece of information, they could add it without worrying about messing with the "record size".

On a side note, a technology getting much hype today is called XML. XML is considered a great leap forward because each record can have whatever information it wants. While there are several plans to force the data to be structured so it can be used with relational databases, the usual answer is to let the data be what it wants and let the conversion software worry about adding missing fields and ignoring extra ones. The concept is the one that Lotus Notes has been using for well over a decade.


IBM has announced that they want to (eventually) replace the data storage in Lotus Notes from its current document-based system to the relational database system used in DB2. While this is good for IBM, because they will sell more DB2 and the programmers to figure out how to make it work, it is bad for the corporate world because it will destroy the one platform that could model data based on reality.

Another problem with using DB2 is that is was designed for mainframes. Lotus Notes allows any application that does not need real-time access to a central data storage to be copied to any computer, with a mechanism to keep both copies updated. I have not heard how IBM plans to keep this functionality using DB2.

The second change they have announced is to push developers to use J2EE. For those outside the industry, J2EE is a set of standards for developing programs in Java. Standards are typically designed so that those who understand them can stay employed. This is a problem for our Lotus Notes Application Developers for several reasons.

The first reason is that the standard requires files to be placed in several different places. Notes applications are typically a single file, so this already adds some confusion. Any change from this single file per application style must impact the ability to easily update and carry the information on multiple systems.

Second is that Java is an "Object Oriented Programming Language". As such it is state-of-the-art for programmers, but many of our former managers and secretaries have little understanding of the advanced programming skills required to know what "Object Oriented" means, never mind use it efficiently, if they can use it at all.

Third is because Java is not a scripting language. All programming in Lotus Notes has been done by finding the obvious place to do something in the interface, and adding just enough code to get it done. Java requires that the code creates the interface. Instead of looking at a form, finding the FirstName field, and adding a line that makes certain it is not blank when the form is saved, the Java programmer has to write code to create the form. Obviously maintenance is much harder, but creating the form may easily take 10 times as long.


IBM is attempting to merge its products, so it can sell more of them. It is attempting to set the new standards, so it can regain its place as one of the drivers of information technology. It is forcing these concepts on what was formerly the independent product know as Lotus Notes.

There are some obvious "Not Invented Here" thoughts about Lotus Notes. The creators sold it to Lotus to gain better distribution. Lotus was bought by IBM because they had products that IBM needs to be competitive. Obviously a company that could be bought cannot be as good as the never-bought IBM. But in each case there were compelling reasons to join with a larger, not better, company.

IBM should focus on the future through looking at its past. Lotus Notes is already a wide-spread, well-respected, and unique product. It was not designed for e-mail, and yet it stays neck-and-neck with Microsoft's dedicated mail software. It was designed before the internet existed, yet it has become one of the better web servers. Lotus believed in the strategy of including every new technology with Lotus Notes. And that has made it the choice for corporations that need to build applications quickly to stay competitive in today's marketplace.

The new strategy if cutting Lotus Notes into many products and merging them with IBM's obsolete product lines was not designed for the future. While Lotus Notes will keep evolving, the product already belongs in a future where applications are easily created by managers and secretaries, the administration of a global email network is handled by a couple of employees who work from home, and the sales force carries all pertinent data to every meeting without worrying what is pertinent to THIS meeting. This is what Lotus Notes does today, and, except for some short-timers at IBM, that is what we all want for tomorrow.

<< Ubiquitous Lotus NotesDomino does not require experts >>

Contact Solprovider
Paul Ercolino