Calendar:ITIP and iMIP Support

Meeting Scheduling in Lightning Proposal (iMip and iTip support)

Current iTIP-iMIP design

A common problem among users of any calendaring software is scheduling shared events, or meetings between people. In general, there should be a natural method to schedule these events that works well within the current paradigms of the software. The RFC specifications for iTIP and iMIP address this. This document will confine itself to our proposed implementation, for questions about the specifications, please refer to these RFC's. Also, there are various diagrams throughout, and in those diagrams, Thunderbird components appear in blue while Lightning components (either existing or proposed) appear in yellow. However, this distinction has nothing to do with which components need to be extended in order to fully support iTIP and iMIP.

Current Implementation

Generally, MIME handling for text/calendar content types happens in two phases. For simplification, I will call the first phase the presentation phase and the second the handling phase. The current iTIP implementation in Lightning involves three components: nsISimpleMimeConverter, ltnMimeConverter, and calItipProtocolHandler.

Presentation Phase

This phase realizes that the user has clicked on an email message with the text/calendar content type. It's goal is to present the message in some way so that the user realizes that this is no mere email message. Instead, it is presented as a simple HTML form.

Handling Phase

The handling phase is instigated by the user interacting with the active content that is generated by the presentation phase. In the current implementation, this means that the user clicked the “Accept�? button. Once this happens, it is translated to a moz-cal-handle-itip:/// URI, which is handled by the registered protocol handler in calItipProtocolHandler. The protocol handler is responsible for parsing the URI (which encapsulates the VCALENDAR data) and in our current implementation, adds the VCALENDAR item to the first calendar in the calICalendarManager's list of calendars.

Note: In order to see this for yourself in Lightning, you will need a
      development release of Thunderbird and you need to add the 
      following preference to your <thunderbird install>/greprefs/all.js
      preference file, or else the security subsystem will not allow the
      external protocol handler to handle the protocol:
      pref("network.protocol-handler.expose.moz-cal-handle-itip", true);

New Proposal

The current functionality only supports adding a VEVENT to a calendar, and only to the first calendar in the list. This is an excellent beginning, but mature iMIP and iTIP support will need to take this much further. Fortunately, we can reuse all the above components, extending them for our needs. For simplicity's sake, I have broken the new proposal into the two phases mentioned earlier: presentation and handling.

New Presentation Phase

The user receives and opens an iMIP compliant, text/calendar content type MIME encoded message through email. iMIP messages include the iTIP method specification as an extra parameter on the content-type. For example, a message containing a meeting request will specify: “Content-Type:text/calendar; method=request;�? as its content type.

If this method is present, then iMIP defines several security and anti-spoofing features that we should adhere to. These items are relatively simple items such as ensuring that the sender of the message matches the sender as specified in the iTIP portion of the message, that the method matches etc. I believe that we will not need to do true iCAL parsing of the iTIP portion of the message in order to satisfy these security requirements. The MIME parser will call the “GatherLine�? function of nsISimpleMimeConverterStub.cpp for each line of the VCALENDAR file. For security checks, we can watch for lines with properties that we are interested in verifying (such as ORGANIZER), and set a flag for each security check that we clear as we parse the VCALENDAR text.

Using a different parses for security is a very bad idea. You have to maintain two parsers, which means that it's more likely to have bugs. Thinking "I don't need all the information, I can use a simpler" parser will fail, because the parser will be nearly as complex. You have to taking escaping into account, the place in the document, etc. So: just use the normal parser. --mvl 13:28, 22 March 2006 (PST)

Next, in the EndGather function, before the “ConvertToHTML�? function is called on our ltnMimeConverter implementation, we will call a new method on that interface that writes the value of the security flag to the ltnMimeConverter object. Therefore, when ConvertToHTML is called, the ltnMimeConverter will know whether or not the message has passed iMIP security. This is important because the iMIP specification states that if the message does not pass the security checks, it should be disregarded.

A generalized sequence diagram for this model is as follows:

Once the ltnMimeConverter is called, its job is to properly format the raw VCALENDAR data and present that data to the user, allowing the user to choose among appropriate actions in response to the method specified in the iTIP component. For example, the appropriate responses to a “REQUEST�? could be presented to the user as “Accept, Decline, Tentatively Accept, Reschedule�?. (See Calendar:iTIP_iMIP_Use_Cases). A different iTIP method will have a different set of appropriate actions, and should present a different set of options to the userthe appropriate actions will change.

Due to limitations of an HTML-only display, we wish to consider overriding the normal “message�? display and using a new Lightning XUL window specifically tailored to meeting requests. I envision one XBL/XUL window that would change based on what type of iTIP message it is displaying and the responses that the user may choose from. This window should be similar to the Edit Event window so that it will remain intuitive to users of Lightning, and it should only display as much data is needed. Please refer to Calendar:iMIP_XUL_GUI_Design for mock-ups and discussions of this UI.

In this XUL window (or in a follow-up window), the user must choose:

1.Where applicable, to the iTIP method in question, the user should be
  prompted to decide what calendar to add the VCALENDAR item to. There  
  should be a preference so that the user can define a default calendar.
2.Where applicable to the iTIP method, the user must choose whether or not
  to inform the organizer/attendee of their choice via a text/calendar MIME
  encoded iTIP message. The user should optionally have the ability to edit
  this message before sending it.

Now, there are some restrictions in the Thunderbird MIME handling model that we would not want to buck. For instance, this XUL window needs to be launched after the MIME handling is completed (otherwise we run the risk of interfering with the connection to the mail server). Therefore, launching the XUL window directly from the ltnMimeConverter may not be the proper place to do it. A better place may be the mailWindowOverlay.js::OnMsgLoaded callback. By adding the XUL handling here, we can also edit the UI controls in Thunderbird to ensure that the message displays properly (for instance, marking the message as read after displaying the XUL window, or changing the way the iMIP message is displayed in the thread pane).

We should not limit the ability of future Thunderbird clients from implementing their own “text/calendar�? content handling (separate from Lightning). Therefore, in the OnMessageLoaded function, the code will attempt to instantiate a lightning specific XPCOM object that contains the iTIP message and iTIP state which was generated by the ltnMimeConverter. This XPCOM object will be called calIMIPListSvc. It will contain a list of iTIP information generated by ltnMimeConverter, referenced by the message-ID from the original email message. If the call to instantiate this object fails, then the OnMessageLoaded function will know that Lightning is either not installed or not working, and it will simply continue.

The XUL Window will be instantiated from the calIMipListSvc object.

This interaction is sketched in the following sequence diagram.

New Handling Phase

Once the item is presented to the user, and the user makes their decision on how to respond to the item, it falls to the iTIP handler to ensure that the proper action is taken, and that the proper responses are generated. While it may be redundant to have an iTIP protocol handler when the XUL presentation layer will have all the necessary information to complete the iTIP response, it maximizes our extensibility. Encapsulating this logic in a non-GUI iTIP protocol handler permits us to automate any portion of the iTIP handling. For instance, most users would probably prefer the application automatically either respond to or decline Free/Busy requests.

A “moz-cal-handle-itip:///�? URI is generated that encapsulates the following items:

1. The calendar Data (which is still escaped–the protocol handler will
   be responsible for unescaping it.)
2. The target calendar to add the item to.
3. Whether the user wishes to edit any response before it is sent to
   the Organizer.

Currently, the moz-cal-handle-itip URI is generated via an HTML Form submit action. We may choose to move away from this model and create our own internal moz-cal-handle-itip URI handler. This would mean that the iTIP URI would be handled natively by the mozilla system in much the same way that a “chrome�? URI is handled. This iTIP URI Handler XPCOM component would be a thin layer and would pass the URI directly to the underlying calITipProtocolHandler subsystem. If we decide to make the iTIP URI into a natively supported URI then we would remove the HTML-ness from the current URI specification, removing the &'s and the ?'s. We could also provide a parser directly,and the calITipProtocol handler would not need to parse the URI itself. Additionally, a native iTIP URI handler is also attractive because it could be used by the XUL iMIP presentation layer thus encapsulating the iTIP protocol's complexity.

A typical URI will look like a standard HTML encoded form URI, for example:

moz-cal-handle-itip:///?preferredCalendar=<calendarIdentifier>&autoRespond=<true|false|no>&data=<escaped VCALENDAR data>

The iTIP handler will be able to distinguish between a new event or an update to a previous event based on the rules specified in the iTIP RFC. It will be responsible for notifying the user if there is an error.

Furthermore, if the autoResponse field is “false�?, then the iTIPProtocol handler will instantiate a compose window containing the proper iTIP response item, and pre-populating the “TO�? field with proper person. The user will be allowed to edit the email message before it is sent. If the autoResponse field is “true�? then the email will be formulated and sent automatically, and the user will not be given the opportunity to edit the message. If the user decides not to send a response to the iTIP message, then the autoResponse field will be “no�?. Note that these three strings are NOT localizable and this URI should NEVER be displayed to the user.

Much of the code for generating a text/calendar encoded email on the fly still lives in the Sunbird codebase file calendarMail.js, and this functionality could be extracted and used by the iTIP protocol handler, or perhaps from an iTIP protocol handling object (if needed).

The general sequence diagram of iTIP response handling would look like this:

VFREEBUSY and VJOURNAL Considerations

In general, these components would work in the same methodology as outlined above (which are described in more VEVENT and VTODO specific terms). Because the allowable methods on VFREEBUSY and VJOURNAL components are a subset of the allowable methods on VEVENT and VTODO components, the underlying subsystem needed to handle these components is the same. However, the XUL presentation of these components will necessarily be different, and the actions allowed to users would also be different.

Additionally, for VFREEBUSY requests, we may want to allow the user to specify that they would like to automatically answer (or automatically decline) iTIP REQUEST's for Free/Busy status. This way we could make this part of iTIP handling automatic, and the user would not have to respond manually to each Free/Busy request that comes into their Inbox.

Simplified User Interaction Overview

In order to illustrate the entire sequence of events, here is a very simplified diagram, which attempts to show the user's interaction with these components.

Overview of Changes

New Components

Achieving iMIP and iTIP support requires extending the current elements of this support as they exist in Lightning and Thunderbird, and adding three new components.

• calIMipListSvc – XPCOM service component that will maintain a listing of messages with iMIP information, listed by their internal message ID. This list will not be persisted between sessions in case these message ID's change while the user is not logged in. The list will be created the first time a user clicks on a message containing calendar content. We can analyze the benefit of removing the message from the list once the message has been processed.
• iTIP URI Handler – XPCOM component for creating an actual moz-cal-handle-itip URI handler (not unlike the chrome URI handler) inside Thunderbird. The current implementation piggybacks atop the HTML URI handling ability for forms. We may wish to remove this constraint, especially since we will move away from the HTML display model. If this is implemented, then the URI definition's formatting above may change slightly,but the information being passed will not change.
• XUL iTIP presentation window - This is a set of XUL/XBL/Javascript components that will display the iTIP meeting request and allow the user to choose their response to the request. It will be used for all types of iTIP component interactions (PUBLISH, REPLY, REQUEST, COUNTER, CANCEL etc).

Changed Thunderbird Components

• Small extensions must be made (adding a small number of interface methods) to nsISimpleMimeConverter. Specifically, an interface for SetiMIPSecurityFlag, and there will also be a need to set certain MIME header attributes. The header attributes that are obvious to be set are: iMIP method, and Message ID. TODO: Finalize this list
• Add support in mailWindowOverlay.js for launching a XUL iMIP/iTIP meeting request/response window.

Changed Lightning Components

• ltnMimeConverter.js – needs to be extended to add in new nsISimpleMimeConverter interfaces. Also, this layer will have to construct a slightly different URI, especially if we move to native URI support. The nature of the HTML that is returned to the MIME system will also need to change if we move toward using a XUL iMIP presentation window.
• calITipProtocolHandler – will need to be completed and extended to handle more components and more methods. It will also have to implement the intricacies of the iTIP specification.
• Create new ical components in calendar/base. For example, in order to create VFREEBUSY and VJOURNAL support, these base level calItems must be created.
• Existing DTD files will be updated for the new XUL iMIP presentation window.
• Existing (or new) CSS skin files may need to be updated (or created) to handle the new XUL iMIP presentation window.

Strategy of Implementation

Due to the work already completed, we already have the Publish method of iTIP handled (at least receiving a Publish response). However, our current implementation assumes that everything it gets is a Publish Action. There is much to be done, therefore, this work will be implemented in stages to ensure that the overall method works well, and to ensure that we have a functional product at each step of the way.

1. Create the XPCOM calIMipListSvc object to be used for communication 
   from the ltnMimeConverter to the mail components so that the XUL GUI
   presentation interface work can be begun and performed in parallel.
2. Implement simple iMIP mime security features in the 
   nsISimpleMimeConverter.
3. Implement the iTIP URI handler as a lightweight pass-through object.
4. Implement the iTIP protocol handler REQUEST logic for VEVENTs/VTODOs
   (for new meetings, not updates yet).
5. Implement the XUL presentation for accepting/declining a meeting
   (i.e. iTIP REPLY).
6. Implement the iTIP protocol handler REPLY logic for VEVENTs/VTODOs.

These six items represent a complete sequence of events for the scheduling of an iTIP event (except for the lack of Free/Busy support). Once these six items are implemented and working, we will have an excellent framework that can be extended to include other methods and components.

The next set of items to be implemented will be dependent on having the above work completed. The following items can be addressed in any order.

• Implement VFREEBUSY support (NEEDED: calItem XPCOM component implementation, iTIP protocol handler support, XUL GUI implementation). We should ensure that our Free/Busy implementation is extensible enough to support the Free/Busy URI features referenced in RFC 2739 as well as the VFREEBUSY component specified in RFC 2447. TODO: Are there other methods of obtaining Free/Busy information that must be considered? There should probably be an entire design page devoted to Free/Busy information.
• Implement iTIP method ADD in protocol handler and XUL GUI.
• Implement iTIP method CANCEL in protocol handler and XUL GUI.
• Implement iTIP method COUNTER in protocol handler and XUL GUI.
• Implement iTIP method DECLINECOUNTER in protocol handler and XUL GUI.
• Complete the PUBLISH (so that updates to existing events can be made) method for iTIP and extend the XUL GUI to include this (move away from the inline HTML, in other words).
• Complete the REQUEST method (to include updates, RSVP-TRUE statuses on attendees, etc).
• Implement the iTIP method REFRESH in protocol handler and XUL GUI.
• Define preferences, and their default values to allow user the ability to automate iTIP handling.
• Implement VJOURNAL support (NEEDED: icalItem XPCOM component for VJOURNAL, iTIP protocol handler support for this component, XUL GUI Implementation, also understand how VJOURNALs will be displayed, stored and handled in Lightning).
• Support delegation of user-chosen events through the UI and manifest this during the REQUEST and RESPONSE handling GUIs.
• Handle security/protocol considerations regarding delegated events, such as handling the case where the Organizer receives the response from the delgated calendar user before the Organizer recieves the notice from the attendee regarding the delegation.

Useful Links

Other iTIP and iMIP calendar pages: Calendar:iTIP_iMIP_Use_Cases Calendar:iMIP_XUL_GUI_Design

Please add your comments either here or on the discussion page.