\input texinfo @setfilename excorporate.info @settitle Excorporate Manual @dircategory Emacs @direntry * Excorporate: (excorporate). Exchange Web Services integration for Emacs. @end direntry @copying Copyright @copyright{} 2016 Free Software Foundation, Inc. @quotation Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover, or Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License'' in the Emacs manual. This document is part of a collection distributed under the GNU Free Documentation License. If you want to distribute this document separately from the collection, you can do so by adding a copy of the license to the document, as described in section 6 of the license. All Emacs Lisp code contained in this document may be used, distributed, and modified without restriction. @end quotation @end copying @titlepage @title Excorporate Manual @author Thomas Fitzsimmons @page @insertcopying @end titlepage @contents @node Top @top Excorporate Manual Excorporate provides Exchange Web Services (EWS) support for Emacs. If the Exchange server you access is configured to provide EWS support, then there's an 86% chance that Excorporate will enable you to retrieve your calendar entries from the comfort of Emacs. The 14% failure rate is because authenticating against an Exchange server can be challenging. Accessing an Exchange server through an HTTPS proxy is possible now that @uref{https://debbugs.gnu.org/cgi/bugreport.cgi?bug=10} and @uref{https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35969} are fixed. Kerberos/GSSAPI authentication needs more experimentation. Reports of success or failure of different connection types are welcome, as are patches to enable more of these access scenarios. See @pxref{Reporting Bugs}. @menu * Reporting Bugs:: How to report bugs in Excorporate * Installation:: Getting and installing @code{excorporate}. * Configuration:: Configuring @code{excorporate}. * Usage:: Using @code{excorporate}. * Troubleshooting:: Debugging why a connection failed * API Usage:: Using the API provided by @code{excorporate}. * Index:: @end menu @node Reporting Bugs @chapter Reporting Bugs @noindent To report a bug, send an email to @code{bug-gnu-emacs@@gnu.org} using your favourite email program. Put ``Excorporate'' somewhere in the subject line, for example: ``Excorporate: Failed to authenticate''. @node Installation @chapter Installation Excorporate works on Emacs versions >= 24.1. @noindent Install @code{excorporate} from the GNU ELPA repository: @code{M-x package-install RET excorporate} @node Configuration @chapter Configuration @noindent Ideally you won't need to configure Excorporate beyond providing your account email address. On friendly Exchange setups, Excorporate can discover the EWS URL automatically. @noindent Run: @code{M-x excorporate} @noindent which will prompt you for the Exchange account configuration. Follow the prompts and if all goes well, you'll see a message in the minibuffer or in *Messages* saying that the connection is ready. Using the prompts, you can first try with autodiscovery. If autodiscovery runs out of URLs to try, re-run @code{excorporate}, saying 'n' to the autodiscovery attempt, at which point you will be asked for the EWS URL. @noindent To save a working configuration, customize @code{excorporate-configuration}: @code{M-x customize-variable RET excorporate-configuration} @noindent After saving the configuration, try @code{M-x excorporate} again. @noindent If neither autodiscovery nor specifying the EWS URL work, @pxref{Troubleshooting}. To disconnect: @code{M-x excorporate-disconnect} @node Usage @chapter Usage @noindent Excorporate can put entries it retrieves into the Emacs Diary, and use @code{appt} to remind you a few minutes before a meeting starts. To enable this support, do: @code{M-x excorporate-diary-enable} Excorporate's diary front-end will retrieve today's meetings. Subsequently @code{appt} will pop up a reminder window several minutes prior to each meeting. If you leave Emacs running overnight, at 12:01 AM @code{appt} (via Excorporate) will retrieve your meetings and display your diary so that you see the day's events first thing in the morning. @noindent Open the calendar with: @code{M-x calendar} @noindent move the cursor to the date you want to see meetings for, and press `d'. Some time later, asynchronously, a window will pop up containing events retrieved from the Exchange server in addition to locally-entered diary events. The events are all sorted by time. Excorporate also binds `e' in @code{*Calendar*} buffers to @code{excorporate-calendar-show-day-function} to allow a different view of retrieved events. By default, @code{excorporate-calendar-show-day-function} is set to @code{exco-org-show-day} which displays meetings in a temporary read-only Org Mode buffer named @code{*Excorporate*}. In the Org Mode @code{*Excorporate*} buffer, you can run @kbd{M-x exco-org-decline-meeting-request} to decline a meeting request. To accept, use (@code{exco-org-accept-meeting-request}) or, to tentatively accept, invoke (@code{exco-org-tentatively-accept-meeting-request}). Pass a prefix argument to these functions to omit a reply message. A meeting is a calendar event to which at least one other person is invited. To cancel a meeting (or an occurence of a recurring meeting) that you organized, use @kbd{M-x exco-org-cancel-meeting}. An appointment is a calendar item that has no invitees. To delete an appointment that you created, type @kbd{M-x exco-org-delete-appointment}. With a prefix argument, @kbd{M-x exco-org-delete-appointment} can be used to force-delete calendar items, whether they be meetings or appointments. One example where this is necessary is when ``cancelling'' a meeting with a single invitee, you, the organizer. The server will reject an attempt to cancel such a meeting because it refuses to send the organizer a cancellation message. If you prefer, you can install the @code{calfw} package, and set @code{excorporate-calendar-show-day-function} to @code{exco-calfw-show-day}. @node Troubleshooting @chapter Troubleshooting @noindent First, you'll want to double-check that the Exchange server you're trying to access provides EWS support. If it doesn't, Excorporate can't do anything for you. Before asking your Exchange administrator, check intranet wikis and so forth; other users of non-standard clients may have already found the EWS URL. This is called the ``EWS endpoint''. It can be as simple as, e.g.: @code{https://mail.gnu.org/EWS/Exchange.asmx} @noindent First you need to make sure you can access the endpoint. @noindent For Exchange Web Services (EWS) which Excorporate uses, you'll have to determine the EWS endpoint for your Exchange account, call it @code{ews-url}. It is usually something like: https://<mail host name>/EWS/Exchange.asmx @noindent Excorporate calculates the WSDL URL, call it @code{wsdl-url}, by replacing the endpoint's last path element with ``Services.wsdl'': https://<mail host name>/EWS/Services.wsdl @noindent Before even attempting Excorporate, you have to make these succeed: @example (with-current-buffer (url-retrieve-synchronously ews-url) (buffer-string)) @end example @noindent When this works, you'll see web page text in *Messages*, containing a message about having created a service. @example (with-current-buffer (url-retrieve-synchronously wsdl-url) (buffer-string)) @end example @noindent When this works, it will show a bunch of WSDL (XML) in *Messages*. @noindent Debug the above URL retrievals with @code{M-:} in an @code{emacs -Q} run: @example (progn (setq url-debug 1) (url-retrieve-synchronously URL-STRING) (dolist (p (seq-filter (lambda (b) (string-match " *http*" (buffer-name b))) (buffer-list))) (message "HTTP result buffer: \"%s\"\n%s" (buffer-name p) (with-current-buffer p (buffer-string)))) "check *Messages*") @end example @noindent Beware that HTTP responses can be out-of-order, and that if you set @code{url-debug} to a number or @code{t}, Emacs may hang for a while if it attempts to print a very large data structure. @noindent Once you're sure the above steps are working, try @code{M-x excorporate}. @noindent The buffer @code{*fsm-debug*} shows @code{excorporate} state transitions and should provide details of where things went wrong. @noindent Also check @code{*Messages*} for anything obvious. @noindent If you suspect something wrong with accessing the EWS URL, try setting @code{url-debug} to t and retry @code{M-x excorporate}, then check the @code{*URL-DEBUG*} buffer for output. @noindent If you suspect NTLM authentication is failing, as a long shot, you might try setting @code{ntlm-compatibility-level} to 0 and retrying @code{M-x excorporate}. @noindent Excorporate's dependencies implement the tricky elements of asynchronous Exchange access: a state machine (@code{fsm}), TLS negotiation (@code{gnutls}), NTLM authentication (@code{ntlm} and @code{url-http-ntlm}) and SOAP communication (@code{soap-client}). @cindex hung connection @cindex stuck connection @noindent On some servers, an active, otherwise-working connection may get stuck. The symptom is the attempted operation will not complete (but Emacs will not be blocked, because Excorporate is asynchronous). For example pressing 'e' in the @code{Calendar} produce a new @code{*Excorporate*} buffer that stays empty for longer than one minute. I haven't been able to determine the root cause of this behaviour. But you can work around the issue like this: @code{M-x list-processes} Find a line that shows the server connection. There may be multiple such lines. They will look something like this: @code{mail.gnu.org -- open -- -- Main (network connection to mail.gnu.org:443} Put the cursor on that line, and type 'd' to delete the process. The attempted operation will now complete, usually without needing to retry it. @node API Usage @chapter API Usage @noindent As of version 1.0.0, the Excorporate API is declared stable. The function @code{exco-api-version} will return 0, meaning API version 0. In the unlikely event that compatibility needs to be broken, the API version will be incremented. @noindent Here are some examples of using the API (application programming interface) provided by Excorporate. @noindent Not all of Excorporate's functionality is exposed as interactive functions. Here is an example of creating a meeting to which hacker2@@gnu.org is invited, using a non-interactive function provided by Excorporate: @example @group (exco-calendar-item-meeting-create (exco-select-connection-identifier) "Test meeting 1" "Hi,\n\nThis is a test meeting 1.\n\nRegards.\n" (encode-time 0 15 14 23 09 2020) (encode-time 0 0 15 23 09 2020) "Online only" '("hacker2@@gnu.org") nil (lambda (identifier response) (message "%S: %S" identifier response))) @result{} ;; Printed in *Messages*: ("hacker1@@gnu.org" . "https://mail.gnu.org/EWS/Exchange.asmx"): (((ResponseMessages (CreateItemResponseMessage (ResponseClass . "Success") (ResponseCode . "NoError") (Items (CalendarItem (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]M")))))))) @end group @end example @noindent The callback is run asychronously after the server responds, so as not to block Emacs, and the result is what is printed in the @code{*Messages*} buffer. This example assumes the user has already run @kbd{M-x excorporate} to create a connection. @code{exco-select-connection-identifier} will automatically use the sole connection if only one exists. Excorporate fully supports connecting to multiple different servers though (see @code{exco-connection-iterate}) so reusable code that calls Excorporate APIs should not assume just one connection. You can find examples of iterating through multiple connections in @code{excorporate-diary.el} and @code{excorporate-org.el}. @noindent There is lots of server-side functionality that Excorporate does not provide high-level non-interactive functions for. Using that functionality is still possible with the low-level @code{exco-operate} and @code{exco-operate-synchronously} functions. @noindent For example, evaluating this form produces lots of details about the meeting represented by the ItemId form, including tidbits like the list of invitees and how they've responded (accepted, declined, tentatively accepted, unknown). You can find ItemId forms to experiment with in the PROPERTIES drawer of calendar entries in the interactive Org buffer. @example @group (exco-operate-synchronously (exco-select-connection-identifier) "GetItem" '(((ItemShape (BaseShape . "AllProperties")) (ItemIds (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]d")))) nil nil nil nil nil nil)) @result{} (((ResponseMessages (GetItemResponseMessage (ResponseClass . "Success") (ResponseCode . "NoError") (Items (CalendarItem (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]M")) (ParentFolderId (Id . "A[...]A") (ChangeKey . "A[...]A==")) (ItemClass . "IPM.Appointment") (Subject . "Excorporate discussion") (Sensitivity . "Normal") (Body (BodyType . "Text") . "Hi Hacker Two, Let's discuss Excorporate. Hacker One") (DateTimeReceived . "2020-09-24T20:07:26Z") (Size . 13709) (Importance . "Normal") (IsSubmitted) (IsDraft) (IsFromMe) (IsResend) (IsUnmodified) (DateTimeSent . "2020-09-24T20:07:26Z") (DateTimeCreated . "2020-09-24T20:07:26Z") (ResponseObjects (ForwardItem) (CancelCalendarItem)) (ReminderDueBy . "2020-09-25T14:30:00Z") (ReminderIsSet . t) (ReminderMinutesBeforeStart . 15) (DisplayCc) (DisplayTo . "Hacker Two") (HasAttachments) (Culture . "en-US") (Start . "2020-09-25T14:30:00Z") (End . "2020-09-25T15:30:00Z") (IsAllDayEvent) (LegacyFreeBusyStatus . "Busy") (Location . "Online") (IsMeeting . t) (IsCancelled) (IsRecurring) (MeetingRequestWasSent . t) (IsResponseRequested . t) (CalendarItemType . "Single") (MyResponseType . "Organizer") (Organizer (Mailbox (Name . "Hacker One") (EmailAddress . "hacker1@@gnu.org") (RoutingType . "SMTP"))) (RequiredAttendees (Attendee (Mailbox (Name . "Hacker Two") (EmailAddress . "hacker2@@gnu.org") (RoutingType . "SMTP") (MailboxType . "Mailbox")) (ResponseType . "Accept") (LastResponseTime . "2020-09-24T21:08:54Z"))) (Duration . "PT1H") (TimeZone . "(UTC+00:00) Monrovia, Reykjavik") (AppointmentSequenceNumber . 0) (AppointmentState . 1) (IsOnlineMeeting))))))) @end group @end example @noindent Note that this function queries the server synchronously. In other words, it waits for, and evaluates to, the server's reply. This is nice when experimenting with the API, but published code should mostly use the asynchronous calls to avoid blocking Emacs during server operations. @noindent Here is a more complicated example that asynchronously queries the server for availability overlap for hacker1@@gnu.org and hacker2@@gnu.org, in the America/Toronto time zone. Call @code{exco-time-zone} to calculate, from Emacs's internal time zone (see @code{current-time-zone}), the equivalent server time zone string. @example @group (exco-operate (exco-select-connection-identifier) "GetUserAvailability" '(((TimeZone (Bias . 300) (StandardTime (Bias . 0) (Time . "02:00:00") (DayOrder . 1) (Month . 11) (DayOfWeek . "Sunday")) (DaylightTime (Bias . -60) (Time . "02:00:00") (DayOrder . 2) (Month . 3) (DayOfWeek . "Sunday"))) (MailboxDataArray (MailboxData (Email (Address . "hacker1@@gnu.org")) (AttendeeType . "Required") (ExcludeConflicts . nil)) (MailboxData (Email (Address . "hacker2@@gnu.org")) (AttendeeType . "Required") (ExcludeConflicts . nil))) (FreeBusyViewOptions (TimeWindow (StartTimeZone (Id . "Eastern Standard Time")) (StartTime . "2020-09-25T00:00:00Z") (EndTime . "2020-09-25T23:59:00Z")) (MergedFreeBusyIntervalInMinutes . 60) (RequestedView "DetailedMerged"))) nil nil nil) (lambda (identifier response) (message "%S: %S" identifier response))) @result{} ;; Printed in *Messages*: ("hacker1@@gnu.org" . "https://mail.gnu.org/EWS/Exchange.asmx"): (((FreeBusyResponseArray (FreeBusyResponse (ResponseMessage (ResponseClass . "Success") (ResponseCode . "NoError")) (FreeBusyView (FreeBusyViewType "FreeBusyMerged") (MergedFreeBusy . "000000000000000000000200") (CalendarEventArray (CalendarEvent (StartTime . "2020-09-25T12:00:00") (EndTime . "2020-09-25T12:30:00") (BusyType . "Busy"))) (WorkingHours (TimeZone (Bias . 480) (StandardTime (Bias . 0) (Time . "02:00:00") (DayOrder . 1) (Month . 11) (DayOfWeek . "Sunday")) (DaylightTime (Bias . -60) (Time . "02:00:00") (DayOrder . 2) (Month . 3) (DayOfWeek . "Sunday"))) (WorkingPeriodArray (WorkingPeriod (DayOfWeek "Monday" "Tuesday" "Wednesday" "Thursday" "Friday") (StartTimeInMinutes . 540) (EndTimeInMinutes . 1080)))))) (FreeBusyResponse (ResponseMessage (ResponseClass . "Success") (ResponseCode . "NoError")) (FreeBusyView (FreeBusyViewType "DetailedMerged") (MergedFreeBusy . "000000000000002200000200") (CalendarEventArray (CalendarEvent (StartTime . "2020-09-25T05:30:00") (EndTime . "2020-09-25T06:30:00") (BusyType . "Busy") (CalendarEventDetails (ID . "0[...]0") (Subject . "Excorporate discussion") (Location . "Online") (IsMeeting . t) (IsRecurring) (IsException) (IsReminderSet . t) (IsPrivate))) (CalendarEvent (StartTime . "2020-09-25T12:00:00") (EndTime . "2020-09-25T12:30:00") (BusyType . "Busy") (CalendarEventDetails (ID . "0[...]0") (Subject . "An occurence of a recurring meeting") (Location) (IsMeeting . t) (IsRecurring . t) (IsException) (IsReminderSet . t) (IsPrivate)))) (WorkingHours (TimeZone (Bias . 480) (StandardTime (Bias . 0) (Time . "02:00:00") (DayOrder . 1) (Month . 11) (DayOfWeek . "Sunday")) (DaylightTime (Bias . -60) (Time . "02:00:00") (DayOrder . 2) (Month . 3) (DayOfWeek . "Sunday"))) (WorkingPeriodArray (WorkingPeriod (DayOfWeek "Monday" "Tuesday" "Wednesday" "Thursday" "Friday") (StartTimeInMinutes . 480) (EndTimeInMinutes . 1020))))))))) @end group @end example @noindent This example shows how to create a recurrence in the ``Eastern Standard Time'' time zone. The @code{exco-operation-arity-nils} call returns a list of nils with a length matching the number of arguments that the @code{CreateItem} operation takes. Arguments other than the first (``request'') argument may be needed in the future to use more complicated server functionality, but for now they can all be left @code{nil}. @example @group (exco-operate (exco-select-connection-identifier) "CreateItem" `(((SendMeetingInvitations . "SendToAllAndSaveCopy") (Items (CalendarItem (Subject . "Test recurrence 1") (Body (BodyType . "Text") "Testing recurrence creation.") (Start . "2020-09-25T17:00:00-04:00") (End . "2020-09-25T18:00:00-04:00") (StartTimeZone (Id . "Eastern Standard Time")) (EndTimeZone (Id . "Eastern Standard Time")) (Location . "Online") (RequiredAttendees (Attendee (Mailbox (EmailAddress . "hacker1@@gnu.org")))) (Recurrence (WeeklyRecurrence (Interval . 1) (DaysOfWeek "Friday")) (NumberedRecurrence (StartDate . "2020-09-25-04:00") (NumberOfOccurrences . 4)))))) ;; Empty arguments. ,@@(cdr (exco-operation-arity-nils identifier "CreateItem"))) (lambda (identifier response) (message "%S: %S" identifier response))) @result{} ;; Printed in *Messages*: ("hacker1@@gnu.org" . "https://mail.gnu.org/EWS/Exchange.asmx"): (((ResponseMessages (CreateItemResponseMessage (ResponseClass . "Success") (ResponseCode . "NoError") (Items (CalendarItem (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]k")))))))) @end group @end example @noindent Now we can retrieve the item's properties to see the recurrence and time zone details: @example @group (exco-operate (exco-select-connection-identifier) "GetItem" '(((ItemShape (BaseShape . "AllProperties")) (ItemIds (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]d")))) nil nil nil nil nil nil) (lambda (identifier response) (message "%S: %S" identifier response))) @result{} ;; Printed in *Messages*: ("hacker1@@gnu.org" . "https://mail.gnu.org/EWS/Exchange.asmx"): (((ResponseMessages (GetItemResponseMessage (ResponseClass . "Success") (ResponseCode . "NoError") (Items (CalendarItem (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]h")) (ParentFolderId (Id . "A[...]A") (ChangeKey . "A[...]A==")) (ItemClass . "IPM.Appointment") (Subject . "Test recurrence 1") (Sensitivity . "Normal") (Body (BodyType . "Text") . "Testing recurrence creation.") (DateTimeReceived . "2020-09-26T00:23:59Z") (Size . 13636) (Importance . "Normal") (IsSubmitted) (IsDraft) (IsFromMe) (IsResend) (IsUnmodified) (DateTimeSent . "2020-09-26T00:23:59Z") (DateTimeCreated . "2020-09-26T00:23:59Z") (ResponseObjects (ForwardItem) (CancelCalendarItem)) (ReminderDueBy . "2020-10-02T21:00:00Z") (ReminderIsSet . t) (ReminderMinutesBeforeStart . 15) (DisplayCc) (DisplayTo . "Hacker One") (HasAttachments) (Culture . "en-US") (Start . "2020-09-25T21:00:00Z") (End . "2020-09-25T22:00:00Z") (IsAllDayEvent) (LegacyFreeBusyStatus . "Busy") (Location . "Online") (IsMeeting . t) (IsCancelled) (IsRecurring) (MeetingRequestWasSent) (IsResponseRequested . t) (CalendarItemType . "RecurringMaster") (MyResponseType . "Organizer") (Organizer (Mailbox (Name . "Hacker One") (EmailAddress . "hacker1@@gnu.org") (RoutingType . "SMTP"))) (RequiredAttendees (Attendee (Mailbox (Name . "Hacker One") (EmailAddress . "hacker1@@gnu.org") (RoutingType . "SMTP") (MailboxType . "Mailbox")) (ResponseType . "Unknown"))) (Duration . "PT1H") (TimeZone . " (UTC-05:00) Eastern Time (US & Canada)") (AppointmentSequenceNumber . 0) (AppointmentState . 1) (Recurrence (WeeklyRecurrence (Interval . 1) (DaysOfWeek "Friday")) (NumberedRecurrence (StartDate . "2020-09-25-04:00") (NumberOfOccurrences . 4))) (FirstOccurrence (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]h")) (Start . "2020-09-25T21:00:00Z") (End . "2020-09-25T22:00:00Z") (OriginalStart . "2020-09-25T21:00:00Z")) (LastOccurrence (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]h")) (Start . "2020-10-16T21:00:00Z") (End . "2020-10-16T22:00:00Z") (OriginalStart . "2020-10-16T21:00:00Z")) (MeetingTimeZone (TimeZoneName . "Eastern Standard Time") (BaseOffset . 0) (Daylight (TimeZoneName . "Daylight") (Offset . 0) (RelativeYearlyRecurrence (DaysOfWeek . "Sunday") (DayOfWeekIndex . "Second") (Month . "March")) (Time . "02:00:00")) (Standard (TimeZoneName . "Standard") (Offset . 0) (RelativeYearlyRecurrence (DaysOfWeek . "Sunday") (DayOfWeekIndex . "First") (Month . "November")) (Time . "02:00:00"))) (IsOnlineMeeting))))))) @end group @end example @noindent Finally, this is how to delete all the occurrences in the series. ItemId here is the top-level recurrence item identifier which is returned as @code{(CalendarItem (ItemId ...) ...)} by the above @code{GetItem} operation, whose @code{CalendarType} element is ``RecurringMaster''. @example @group (exco-operate (exco-select-connection-identifier) "DeleteItem" '(((DeleteType . "MoveToDeletedItems") (SendMeetingCancellations . "SendToNone") (ItemIds (ItemId (Id . "A[...]A==") (ChangeKey . "D[...]h")))) nil nil nil) (lambda (identifier response) (message "%S: %S" identifier response))) @result{} ;; Printed in *Messages*: ("hacker1@@gnu.org" . "https://mail.gnu.org/EWS/Exchange.asmx"): (((ResponseMessages (DeleteItemResponseMessage (ResponseClass . "Success") (ResponseCode . "NoError"))))) @end group @end example @noindent Feel free to contribute new functions that you think others would find useful; file a bug with a patch against @code{https://git.savannah.gnu.org/git/emacs/elpa.git}. Functions in @code{excorporate.el} must always keep the same interface so that they stay backward compatible. If an existing function has an insufficient interface, make a new one. Excorporate functions are written to work with older Emacs versions, back to Emacs 24.1. @node Index @unnumbered Index @printindex cp @bye