Tuesday, January 22, 2008

Lotusphere 2008 - Day 2: Monday, January 21st

8:00 AM - Opening General Session

This has been live blogged, pulled apart, and covered from every possible angle. All I'm going to do is pull out the bits that interested me.
  • Last year Lotus made a number of announcement at Lotusphere and shipped all of them by mid-year.
  • There was a 9% revenue growth. It's more interesting what was not said. Last year it was "X quarters of consistent gains".
  • The guest speaker, Bob Costas, was lame. He made some pretty disgusting comments about the Chinese government. I'm not going to go any further with this; lots of stuff is circling in the rumor mill. I can neither confirm nor deny any of it.
  • Ubuntu will be supported for the Notes client after 8.01.
  • Domino 8 offers 35% savings in file storage and 50% CPU efficiency. DWA is 45% faster.
  • DWA 8.(something) will include Google map integration and spell checking.
  • 8.0.1 will be released (in some form, possibly a public beta) in February 2008.
  • Domino 8.5 is going to @#$%^ rock!! ID files can be stored in a server vault and the Notes client can retrieve the ID at runtime. NO MORE MANAGING ID'S!! You don't have to run Domino Directory, you can opt to use others. You can offload attachments to an external store.
  • Domino Designer 8.5 is T H E best announcement. (More on that in a bit).
  • I slept through Quickr announcements. Interpret that however you like. :-)
  • All five Connections services will be available on a Blackberry via a client provied by RIM.
  • The Lotus Foundations box that Mike pulled out of an envelope is a little bigger than a sheet of Letter paper (I dunno, it might be as big as an A4), and about 3"thick. It's a full Domino server. F'ing AMAZING!

2:30 - NSFDB2: Converting an IBM Lotus Domino NSF to IBM DB2 back-end - How, What and Why?

Lance Spellman, Tom Truitt

If you will indulge me, I have a brief rant. Francie Whitlock was unable to present this session for personal reasons and had to fly back to Anguilla early Monday morning. Tom Truitt, a veteran Domino admin, stepped in. Lance took it upon himself to replace Francie's name with Tom's on the session slides. Francie worked on these for months prior to Lotusphere and it was extremely tacky of Lance to take her name off them. Many were her work, and she deserves the recognition for that. All I can do is blog my displeasure, which I doubt will ever make it back to Lance, but I feel better for having done so. Very poor form, sir.

The content of the session was very good. There is a lot to getting NSFDB2 set up, so grab the session slides for a list of part numbers to download from Passport Advantage. They really need to make this a single-executable install. It is interesting to note that multiple Domino servers can connect to the same DB2 server.

It is recommended that you create a separate DB2 instance per Domino server, but it is not required. Be aware that running DB2 on the same server as Domino may actually decrease overall performance. If the Domino server is already resource constrained, adding DB2 isn't going to help. So depending on the reason you're using NSFDB2 in the first place this may not apply. Just be aware of it.

A few caveats about using NSFDB2:
  1. DB2 is organized into instances and databases. Conceptually it is like partitioned Domino servers, where you have one physical server and one install of Domino, but you have multiple instances of Domino and each of those contains multiple Notes databases. When you install DB2 the default instance (partition) is DB2, and that's fine. The problem comes in naming the initial database. You will be tempted to name it the same as the Domino server it is supporting. DON'T. You'll get odd errors that make no sense.
  2. You must enable transaction logging on the server. The best practice is to put the transaction logs on their own set of RAID 1 drives, separate from the OS, OS page file, or Domino. This ensures maximum throughput for the transaction logs. In testing you can put it on the same drives, but it is not recommended to do this in production. There, I've done my "admin good duty". Now maybe Paul will stop beating me with that stupid bat.
  3. ODS 43 is required. If the database you want to convert to NSFDB2 is on ODS 48 (the Domino 8 ODS), you will need to use load compact app.nsf -f to downgrade it to ODS 43.
  4. When working with DB2 Access Views, after you create the view you must use the Update in DB2 and the Update actions before the view will have any rows.
  5. Include #NoteID in your SELECT statements for DB2 Access Views and Query Views and it will open the document referenced by the NoteID when you double-click the entry in the view. If you leave this out, the document won't open.
When you migrate the NSF to DB2 you end up with a 1KB NSF in the Domino Data directory. This just links over to DB2 for all the design elements and data. If you create a replica of a database that has been migrated to DB2 it creates a full NSF with all the design elements and data in it. The indexer task will still run on all views, including DAV's and QV's, but the indexes themselves are stored in DB2.

So, why DB2? The biggest reasons are to reduce the overall number of views, since you can have data from multiple forms to be presented in a single view (via a Query View) or to allow back-end access to Domino data from other systems. How do you include data from multiple forms into a single view? When you create a Query View it has a SELECT statement that is written in SQL. Here is how it translates:

Formula: SELECT (Form = "Invoices" : "Orders") & @Year(DateEntered) = 2007;
SQL: SELECT Orders.*, Invoices.* FROM Orders INNER JOIN Invoices ON Orders.InvNum = Invoices.InvNum

So you're going to have to learn some SQL to work with Query Views and Data Access Views. It's well worth it.

3:30 - Features Every Notes and Domino Application Should Have

Captain Oddball (Julian Robichaux)
Captain BrownBeard (Andre Guirard)

For the most part everything is on the slides, so I won't repeat it here. There is a TON of good content.

  1. Don't force the user to fill in fields in the order they're on the screen. Add @IsDocBeingRecalculated to Input Validation formulas to prevent them from firing while a user is still interacting with the form: @If(@IsDocBeingRecalculated;@Return(@Success);@ThisValue = "";@Failure("No blanks allowed.");@Success).
  2. Use templates. Don't update actual real databases. If something goes really really bad you can revert to a previous design. It's also worth knowing that design elements can be individually inherited from different templates. For example you can have a database of common script libraries and inherit from that, and have a different template with UI pieces, and yet another template with agents.
  3. Use a help form and view, then use @Command([OpenHelpDocument]...) to display it. This will produce a standard help window that can be viewed outside Notes and won't lock the display, and it will let you delegate help authoring to someone else.

5:00 PM - Templates, Templates Everywhere

Bruce Elgort
Kevin Pettitt

This was a great overview of a lot of good templates that are available. These include things available from OpenNTF and individual blogs. I think Kevin said they have over 120 slides and covered nearly 30 templates, so download the presentation to get a glimpse of what they covered. My favorites were:
  • Team Mailbox
  • Design Catalog (un-freaking-believable)
  • Vacation Request (it needs a makeover)
  • Doc Master (by Chris Doig, no link was provided)
  • Domain Patrol
  • Conflict Manager
  • Dolmio
  • OpenSlice
  • Domino Application Framework

6:15 - Blogger Press Conference

This was the second annual event, and I was both surprised and delighted to be invited. As we walked in we were asked to write down our names, which were put in a hat (literally) and drawn to see who would ask questions. There were 25 bloggers attending, and for most of the time it was only Mike Rhodin fielding the questions. Alistair joined him about mid-way through.

Q Chris Miller: What is Lotus doing for Web 3.0?
A Virtual worlds are interesting and still involving, and IBM is experimenting. They are using gaming for training and certification. You get the certification when you get good enough to progress to the next level. SAAS and social software has a long run ahead of it, but IBM is definitely interested in being at the forefront of the next evolution.

Q Russ Mayfield: What is the most disruptive technology that Mike has seen that needs to be fully understood and embraced all the way down the organization?
A Social software. It breaks all the traditional chains of command and control. Businesses don't know where their potentially confidentially information is being shared, and they need help in understanding how to manage this risk while taking advantage of the benefits.

Q Yancy Lents: Was there anything Mike wanted to cover at the OGS that he couldn't due to time.
A This is why Mini Mains were added throughout the conference, to give more time to focus on specific products. In particular there are a lot of things happening with Unified Communications.

Q John Head: Will Foundations include a hosted mail component?
A Hosted Notes was introduced in September 2007 due to market demand. It isn't in Foundations now but that is the next logical step. The ultimate goal is to integrated Foundations and bluehouse in such a way that it is completely invisible to the end user what is on premises and what is delivered via SAAS. There is a need for on premises in developing countries.

Q Rob Novak: Is there visibility to the issues caused by Windows-specific pieces, such as ActiveX controls?
A Yes, and they are rooting them out as they find them. They are very interested in Rob's list of areas that concern him.

Q Rich Schwartz: Is there any commitment by IBM to get into local community colleges? Will there be a certification for the SMB products that were announced?
A [There was no firm answer given to the first part of the question] A certification program for SMB is high on Mike's list.

Q Mitch Cohen: Public social network have Outlook integration. Why not Notes?
A They're currently finding the right approach. They understand it's needed, but with Connections out there some of the social sites consider Lotus a threat. "Watch this space."

Q Bill Buchan: How has Lotus performed versus competitors for the past year?
A Microsoft was the most aggressive they've ever been. When Microsoft takes on someone head on they don't expect them to fight back. They are reeling from the punches, and the last thing they ever expected was Symphony.

Q Bruce Elgort: A new discussion template was presented during the OGS. What about the rest of them? What about having a stable of good, high-value templates available out of the box like Sharepoint has?
A All of the templates will eventually be redesigned using XPages. [There was no good answer given to the last part of the question.]

Q Alexander Kluge: What does IBM see as the role of partners in the new Foundations and bluehouse SAAS offerings?
A Foundations is extended by bluehouse and BP's can play a role in either place. bluehouse is intentionally loosely coupled to allow BP's to integrate into it.

Q Warren Elsmore: Foundations is obviously a play against MS Small Business Server. It is a totally different model for the very small shops than what IBM has traditionally gone for. How is Lotus handling that?
A It's definitely strategic. A lot is going on, Nitix was a tactical acquisition to get to their 2600 resellers and partners. They will be capitalizing on the role of those partners.

Q Matt White: What about Domino Designer on Mac or Linux?
A [There was a lot of noncommittal faff. In short, it's likely to happen in the 8.5 release but there is no guarantee.]

Q Nathan Freeman: How does lotus plan to integrate their current community?
A Greenhouse is out there for everyone to use. Best Practices Wiki's have been launched. These Wiki's will be community editable. [This contradicts what has been said by other IBM'ers, in particular, those responsible for said wiki's. Alistair committed to getting everyone on the same page.]


  1. About DB2NSF:
    What I'm hearing is that this is a botched product. The proof of that is the one platform that might really use it in large numbers, AS/400, isn't supported and that the underlying reason is that the implementation was so bad that it couldn't be made to work on the AS/400. You might check around to see if this has a future before putting a ton of work into it.

  2. Tim, that's not the reason I was given for NSFDB2 not running on the AS/400. I was told it's because DB2/400 is a completely different product. I plan to be in the labs for a bit today, so I'll see if I can get a straight answer from Wai Kai.

  3. You will never get a straight honest answer, It's about revenue.

  4. A few corrections/clarifications...

    1) Naming the DB2 database - DB2 has some odd notions of namespaces, and the safest approach is to have the userid != instancename != databasename != systemname when setting things up. Be careful with collisions with DB2 keywords too (such as NODE, DATABASE, etc...)

    2) ODS 43 is certainly NOT required for DB2NSF. Having said that, there was a bug in 8.0 (fixed in 8.01) that prevented compact -p from completing properly on databases ODS45(?) or newer. Two workarounds: a) convert to DB2NSF as ODS43 and *then* upgrade to newer ODS, or b) temporarily set the server's default creation type to DB2 (restart required) and create a new replica of your ODS48 database, which avoids using compact -p.

    3) QV's - Create in DB2 and Populate needs to be done ONCE before the DB2 tables are available for use. Their data contents are maintained automatically after that.

    4) Another huge strength of QVs is that the SQL statement that the view is based on is "just a string", and can be manipulated *at runtime* to customize the query that is executed. Very flexible...very powerful.

    5) The real reason that AS400 is not supported: this is not a slight against AS400 folk, or any nefarious plot. It's simply that the DB2 engine on AS400 is significantly different than the DB2 engine used in the L-U-W product. The differences are great enough that it became unfeasible to produce an AS400 version. In addition to many subtle (yet crucial) differences in statement handling and processing, the API to build a UDF (such as DB2 Access for Lotus Domino) is completely different, and does not provide all the hooks needed to achieve the same level of functionality. No conspiracy, no revenue presures, just technical reasons.

  5. Correction: #3 above should read "DAVs" (not QVs)

  6. Thanks for the corrections and clarifications. In particular, the presenters were very clear that ODS 43 was required. Could the slides be updated to reflect the correct information?

    Regarding QV's, are you saying that I don't have to click Populate for each view, but rather only after I create the first one. If I add more forms (which creates more tables), do I need to use Populate again? This wasn't at all clear in the presentation.

    Rocky Oliver mentioned @URLQueryString, which he said could be used in 8.5 to customize a QV at runtime. Is there another method available in 8.0 or 8.0.1?

  7. I'm not familiar with the presentation that claimed ODS43... it may be possible to get in touch with the author to correct that, but it may not be. I intend to read it as soon as I get a chance.

    In that presenter's defense, the compact -p bug in 8.0 would have prevented an ODS48 NSF from being converted to DB2NSF...but that was just a bug. It may be a *practical* restriction given the bug, but it's not an intentional one, as 8.01's fixed compact -p or the workarounds will demonstrate.

    The create operation for a DAV creates the tables in DB2 for that DAV. The populate step does the initial population of those tables from data residing in the DB2NSF. (It's effectively optional if you have no data in the DB2NSF.) At a minimum, each DAV needs to go through the create process...just once per DAV. Same for the populate process...once per DAV.

    I'm not positive if the @fn is available in 8.0/8.01, but you can always use @fns (including ones that present dialogs to the user) in the specification for the string property of the QV's SQL query statement.