We use Domino applications to supply data for some of our distributed applications, including Prominic Native.
The external applications retrieve the data through Java agents. These agents perform the requested lookup operation and then return the document data in XML format. There are a lot of agents, so we would prefer to convert them rather than rewriting and retesting the logic.
We use Domino forms to create and edit the stored data internally. Over time, the forms have been updated and expanded until they are quite large and complicated. The forms have hundreds of fields and dozens of subforms.
Why do we want to change?
The Java agents are slow and inefficient, currently. There is very little caching, and iterating through large views can be slow. We could update the agents to do some simple caching, or convert them to XAgents to take advantage of the available caching features, but we decided we would rather change environments if we were going to invest the time to make significant changes to the agents.
We also found that the agents limited how we could supply data to the external applications. For example, we would like to use web sockets to dynamically push data updates to the client applications, but we don’t have a good way to do this using the agent model.
Grails (https://grails.org/) is a web framework based on the Groovy programming language. Groovy (http://www.groovy-lang.org/) is based on Java but includes additional features that make it faster to write code. You can also write pure Java code and run it with Groovy, so it feels familiar to us.
Grails uses the principle of “convention over configuration” to make it easier to quickly build applications. It also has built-in support to make it easy to create simple interfaces. There are also many plugins available to add additional common functionality.
Why a Relational Database?
By moving the data to a relational database, we hope to improve the accessibility and flexibility of our data. With the Domino applications, if a staff member needs a specialized report from the data, a developer would need to create a new view or agent. Once the data is in a relational database, we can create the same report with an SQL query.
We also expect to see a significant performance increase in our lookups, which then make our applications faster.
How is Structure4 Solving These Problems?
The first thing we needed to do was to define how the Domino data would be structured in the new database. We chose to treat the Domino forms as tables for our conversion. Technically, forms have some other features that will not be supported with a table:
– You can add additional items to documents that will not appear in the form. We do not have many cases of this in our application, and we chose to just add a hidden field in the forms for these values if needed.
– You can add use multiple forms to display the same document. This is not a use case for our code, so we have ignored it.
– A form can be used to just input data for an agent without actually storing the data. These forms are not relevant to the data conversion
Grails uses “domain classes” to define the structure of the data. A domain class is a special class designed to map the included fields to a database table using Hibernate. We chose to focus on converting to the Domino forms to Grails domain classes, and let Hibernate take care of configuring the tables.
In order to automate the creation of the domain classes, we created a Grails application that reads the DXL for the forms and parses out the fields. It then displays the list of fields with options to configure each of them in the domain class. This includes:
– Grails field name
– Type – a default is automatically selected based on the Domino type
– Comments – add some documentation to the data model
– Constraints – automatic data validation in Grails
The application automatically generates defaults for the name and type based on the Domino field name and type. It is able to generate a domain class for a form without doing any additional configuration if desired.
Since Domino is not a relational database, the relationships are not explicitly defined in the forms. Our tool allows us to configure relationships between the domain classes by defining the following additional information:
– The referenced form/domain class
– The “key” field from the referenced domain class
The field values are then automatically converted to be references to the target object when importing the data.
Once we had a design for the database, we needed to transfer the Domino data into the relational database. To automate this, we created another Grails application. It expects the following input:
– The generated domain classes for the database. The generated domain classes are populated with metadata that gives the import tool all the information it needs to convert the Domino documents.
– The database to import
– The target relational database
– The views to process for the import.
The import application also allows users to import all documents in the database, but we encourage using views instead. This gives more control over the order in which documents are imported, which is critical for relationships.
The import application can repeatedly run a plan, writing the data to the same database or to a sequence of databases.
Currently, we are only working on logic to transfer data from Domino to Grails. We decided that two-way synchronization of the data would be overkill for our needs, and we did not want to invest the development time.
We have user documents in Domino which also need to be imported to Grails. These forms are configured as special domain classes. We wrote a Grails plugin that will use these domain classes to let users authenticate to a Grails application. This plugin also supports configuring users for alternative authentication methods, including through social media accounts like Facebook and LinkedIN.
We wanted a solution to quickly convert our agents to run in a Grails application using the generated domain classes. Java agents interact with the Domino database through the Domino API, so we had the idea to create our own implementation of the Domino API which would provide similar functionality using the domain classes instead.
Implementing the full API would be overkill for our needs, so we created some tools to analyze our agents for Domino API references. Once we had this list, we focused on implementing these methods first.
However, we also needed to convert the Domino views, since most of the agents accessed the database through views. We identified the following view features that would need to be converted:
– View selection formula – this determines which documents appear in the view
– Column sorting
– Column formulas – this can be a simple field, or a formula. Since our agents generally don’t use the columns, we focused our efforts on the columns used for sorting.
Converting the formulas proved difficult since the formulas can be arbitrarily complex. In addition, some formula functions do not have any meaning in Grails. We focused on iteratively improving the parsing and conversion logic until we were able to convert the views needed for our agents.
In order to do view lookups, we created SQL views within the relational database. These views implement the selection filter and sorted columns, and then contain the information we need to look up the full documents.
Finally, we needed to convert the user interface forms to an interface we could use with the Grails applications. We broke this into two steps.
First, we wrote a tool to convert the Domino forms into Balsamiq Mockups (https://balsamiq.com/). This allows us conveniently redesign the forms for use in the new environment. The mockup includes the metadata added when configuring the domain classes. This allows the mockup controls to be mapped back to the corresponding Domino field.
Then we implemented a tool to convert the Balsamiq Mockups to a simple UI in another language. For our current implementation, we are generating FlexJS code.
A work in Progress
We are still in the process of converting our Domino application, so many of these tools will change before the final version. If you are interested learning more about Structure4, please send an email to Sales@Prominic.NET.
Want to learn more about developing in Grails?