After 1,5 years of development we are happy to announce Z-Push 2.0 final. Some of the new features: - supports ActiveSync version 14, allowing full HTML email synchronization on all capable devices. - possibility to set/unset your out-of-office settings on your mobile (not all devices support this feature) - "z-push-top" tool which shows top-like informations about currently connected devices and their activity - a real real-time notification mechanism for the Zarafa Backend (requires ZCP 7.0.6 or higher) - performs semantic data checks before sending data to a mobile - identifies and fixes or ignores broken items - bi-directional and inter process loop detection - server side search in emails which are not on the phone - meeting organizer sees the status of attendees (accepted/declined/tentatively) on the phone - notes synchronization (only available for some clients e.g. Touchdown on Android) - heartbeat and partial synchronization support - support of recurring tasks (Mantis #125) - global synchronization of the Public Folder (Mantis #198) - improved logging (loglevels, logs on user level possible) - introducing the "combined" backend There is a lot of new code regarding the ActiveSync 14 support. Many synchronization mechanisms have changed. We also switched our Bugtracker. From this version on all tickets will be found at http://jira.zarafa.com/browse/ZP. We would like to thank everybody who contributed with testing and reporting of issues and ideas especially Jan aka deHoeninger, Vincent aka liverpoolfcfan, Erwin aka ebogaard, Uwe aka uwe347, Niels aka nprzybilla, Paul aka pfloor and Bert. We would also like to thank dw2412 – Andreas Brodowski – for his work on the AS12 branch. A few code blocks of his branch made it into the release. Apologies and a big thank you if we have forgotten someone. An updated version of the Mobile Device Management Plugin for the Zarafa Webaccess, which supports Z-Push 2, was also released. It has a new feature which allows resynchronisation of the mobile devices. Please check the Community Hub for more information: http://z-push.sf.net/mdmplugin There is no upgrade path for Z-Push 1.x versions, but as this version implements a fully automatic resynchronizing of devices in the case states are missing or faulty, you should upgrade _without_ maintaining the existing states. The state directory is by default /var/lib/z-push, the log directory /var/log/z-push. Please create these directories and make them writeable for apache or change the configuration parameters to your needs. Be aware that if you are using LOGLEVEL_DEBUG and LOGLEVEL_WBXML Z-Push will be quite talkative, so it is advisable to use log-rotate on the log file. General - Publish updated Mobile Device Management (MDM) plugin (Mantis #586) - implemented: ChangesSink support (Mantis #562) - implemented: Global sync status (Mantis #587) - implemented: Note support (Mantis #550) - implemented: Ability to ignore broken messages already from the backend (Mantis #568) - implemented: Save device information sent through Settings (Mantis #557) - implemented: AS 14 support with basic ItemOperation and Settings - implemented: semantic checks for SyncObjects - implemented: 'z-push-top' functionality - implemented: Detection of concurrent PING connections (Mantis #484) - implemented: Incoming (failstate) and outgoing loop detection (Mantis #410) - implemented: Dynamic WindowSize property per device modified by loop detection (Mantis #452) - implemented: state types to identify default, hierarchy, failsave, devicedata and pingdata states in the statemachine - implemented: HTML e-mail support for Android 2.2+ phones - implemented: Backend, Changes, StateMachine, SearchProvider interfaces (partly Mantis #408) - implemented: 'Supported' (ghosted) properties (Mantis #409) - implemented: recreating a profile on a device or requesting ForceFullResync() removes all related old states - implemented: fail2ban compatibility - writes WARN into the main log - can be enabled with LOGAUTHFAIL - implemented: allow un-authenticated Provisioning commands for remote wipe - implemented: Status handling for FolderSync, Sync, GetItemEstimate, Ping, FolderChange(FolderCreate, FolderUpdate, FolderDelete), Search, GetHierarchy, GetAttachment, MeetingResponse, MoveItems commands - implemented: Status exceptions with HTTPReturnCodeException() for SendMail, SmartForward and SmartReply commands - implemented: Full StatusException handling to Zarafa and Diff backends, basic support in IMAP, Maildir and vCard backends - implemented: SyncProvisioning object for AS12+ provisioning - implemented: use forked php-mapi classes - implemented: refactored generic backend - implemented: better timezone handling (Mantis #479) - thx to Robert Scheck for suggestion and patch - implemented: alternative to apache_request_headers() - implemented: ContentParameters class for SYNC_OPTIONS triggering major improvements on IChanges sub-interfaces and implementing classes - implemented: basic AS12 body preference parsing - implemented: use DeviceManager->GetFolderClassFromCacheByID() to determine content class for AS12+ calls - implemented: Default policies to be applied (Mantis #589) - implemented: GetStateHash() to IStateMachine - implemented: z-push-admin works with Z-Push 2 user API - implemented: Command 'Sync' with 'heartbeat' and 'partial' support - implemented: Email-message flags (red flag aka todo flags) (Mantis #511) - implemented: Objects which can not be synchronized (semantic or loop detection) are saved for users later attention (also in MDM) - implemented: webservices for remote device administration (Mantis #546) - implemented: Save AS version in device data. Used AS version is displayed in z-push-admin (Mantis #551) - implemented: IBackend->GetSupportedASVersion(). The default Backend class returns AS2.5 while backends can overwrite this value if e.g. AS14 functionality is implemented. - implemented: Permanent and State related storage functionality for backends - implemented: [ZP-30] - Search emails on server - implemented: [ZP-31] - Attendee status should be available on phone - implemented: [ZP-65] - Empty server trash - implemented: [ZP-94] - Better log for WBXMLExceptions - added: [ZP-109] - Add GetMailboxSearchResults to the search interface - added: [ZP-115] - Add a destroy method to the ISearchProvider so previous searches can be terminated - added: [ZP-96] - Add segfault detection to process loopdetection - added: [ZP-102] - Add config parameter for changes sink re-export time - added: help information in z-push-top available with 'h' or 'help' - added: HandleGetAttachment() logs and announces information - added: configurable $timeout in HandlePing (Mantis #528) - added: new streamer type STREAMER_TYPE_SEND_EMPTY in order to be able to stream out required empty items with the wbxml streamer - added: conversion functions from command to a constant code and vice versa - added: perfom SyncObject->Check() before streaming object to mobile - added: DecodeBase64URI function for base64 encoded query parsing - added: perform SyncObject->Check() when fetching or when receiving an incoming object - added: definitions for body type of out of office message (related to Mantis #505) - added: try to read SVN information directly to determine version - added: use CommandCodes for command operations - added: additional property deadoccur in order to support tasks iOS5 tasks - added: Command parameters for base64 encoded requests (AS >= 12.1) - added: base64 encoded query string parsing (thx dw2412 for suggestion) - added: new Streamer functionality to support non-container arrays - added: Use mutex when writing to usercache in FileStatemachine (Mantis #588) - added: Option to z-push-admin to reset loop detection (Mantis #608) - added: [ZP-5] - Extend loop detection for invalid folderids in combination with SYNC_STATUS_FOLDERHIERARCHYCHANGED - changed: improved state management (Mantis #163) - changed: Specific backend files are only included if necessary and on request - changed: CombinedBackend includes sub-backends dynamically - changed: re-factored Authentication into RequestProcessor - changed: re-factored BackendDiff internals - changed: general log date & time format changed - changed: configuration check order to improve logging - changed: write device id from Request into the log - changed: do not save devicedata for invalid "validate" device ids sent by Android server checks - changed: check config before starting to parse request data - changed: IBackend->MeetingResponse removing reference to resulting calendar entry, result is returned now. All backends updated. - changed: load backend and statemachine already during ZPush::CheckConfig() so errors could be displayed in browser - changed: renamed BackendVCDir to BackendVCardDir and SearchLDAP to BackendSearchLDAP to be compatible with ZPush::IncludeBackend() - changed: introduced SYNC_PROVISION_RWSTATUS_REQUESTED (intermediary between PENDING and WIPED) - changed: introduced latest known PolicyKey to PING data to push provisioning updates - changed: throw FatalMisconfigurationException if FileStateMachine->SetState() fails (runtime) - changed: IStateMachine gets and saves states as 'mixed' data. Serialization is then done inside FileStateMachine - changed: splitted WBXMLDefs, WBXMLDecoder and WBXMLEncoder in separate classes - changed: use 9 digits for policy key as 10 might exceed the max value for it in base64 encoded string (thx to liverpoolfcfan & dw2412) - changed: IBackend->GetAttachmentData() returns an SyncItemOperationsAttachment object and throws StatusExceptions - adepted all related backends - changed: do not initialize WBXMLDecoder if the input stream is not WBXML (needed for some plain text commands e.g. SendMail at devices which only support AS < 14) - changed: Command 'Ping' uses SyncCollections and does not need a dedicated syncstate any more - changed: Command 'GetItemEstimate' uses SyncCollections - changed: Streamer implements Serializable so SyncObjects can be serialized easily - changed: refactored StateManager from DeviceManager - changed: use StateObject for ASDevice - changed: refactored SendMail, SmartForward and SmartReply commands (Mantis #510) - changed: always use reference PolicyKey from collections to execute ProvisioningRequired() - changed: use latest AS version supported by Z-Push if the mobile does not announce the AS version - changed: Refactored RequestProcessor (Mantis #591) - changed: [ZP-29] - Refactor SyncOptions to support several classes - changed: [ZP-73] - Review truncation definitions - changed: [ZP-72] - Options of Fetch operations should not be saved in the folder data - changed: [ZP-74] - On Fetch requests states don't need to be loaded - fixed: [ZP-87] - PHP's ob_start() has to be initialized with NULL - fixed: [ZP-95] - Impossible to rename folders on Android - fixed: [ZP-99] - Old states are not discarded on Hierarchy operations - fixed: [ZP-103] - Heartbeat exports already processed objects again - fixed: [ZP-104] - SGSII with Android 4.0.3 duplicates objects on every sync/update - fixed: [ZP-106] - z-push-top shows 0 objects synchronized while changes were sent - fixed: [ZP-107] - z-push-top tweaks - fixed: [ZP-57] - Warning when creating an appointment without a reminder - fixed: [ZP-67] - Warning in logfile when marking a mail as read - fixed: [ZP-88] - Provisioning on Nokia E-71 (Mail for Exchange 3.00.73) fails - fixed: [ZP-22] - RIM's Playbook does resend emails over and over again - fixed: [ZP-26] - Ignored messages which are exported again are sent with instead of - fixed: [ZP-37] - Exception if device sends SynckKey "1" on FolderSync - fixed: [ZP-41] - No correct Meeting request invitation displayed on MSExchange when sent from iOS over Z-Push - fixed: [ZP-47] - Warning in FileStatemachine - Invalid argument - fixed: [ZP-55] - From addresses with a not valid FQDN are displayed as broken-from@z-push.local - fixed: [ZP-69] - Loop if ReferencePolicyKey() is empty - fixed: [ZP-70] - Folderlist should not be sent in case of a non OK Ping status - fixed: [ZP-75] - Synchronising with Windows Phone 7.5 causes loop - fixed: [ZP-78] - Errors when doing conflict detection - fixed: Error when executing z-push-top or admin without permissions to log file (Mantis #612) - fixed: GetItemEstimate without valid state should return error status (Mantis #581) - fixed: Usage instructions of z-push-admin can be misinterpreted (Mantis #600) - fixed: Switching from limited sync to full sync changes nothing (Mantis #613) - fixed: Last ping cycle should not cause the request to be longer than the lifetime (Mantis #596) - fixed: Touchdown does not work if provisioning is disabled (Mantis #594) - fixed: Processor does not send with empty sync response (Mantis 566) - thx to liverpoolfcfan for reporting - fixed: WP 7 and WM 6.5 are unable to sync (Mantis #575) - fixed: Impossible to process incoming changes if the content class of the folder is unknown (Mantis #593) - thx deHoeninger for reporting - fixed: Do not use functions deprecated in PHP 5.3 (Mantis #576) - fixed: Emails do not have a sender on phone (Mantis #556) - fixed: Created MR from iOS are not created on the server (Mantis #560) - fixed: organizer name and email are optional elements of SyncAppointment. Do not fail on Check() (Mantis #560) - fixed: set GET user to "unknown" if no username is available (Mantis #575) - fixed: New mail from android produces @localhost error in spooler and postfix logs. Do not parse an email address if it is set but empty. (Mantis #554) - fixed: GalaxyTab sends empty ping folders tag (Mantis #582) – thx to Steve for reporting and fixing - fixed: It's impossible to send an email with an attachment using iPhone with iOS 4 or 5 (Mantis #531) - fixed: typo in SYNC_AIRSYNCBASE_ATTACHMENTS preventing emails with attachments to by synched - fixed: after detecting a loop changes might have gone away (e.g. by deleting them). Loop detection is canceled then (Mantis #521) - fixed: PingTracking: do not timeout connection if it would terminate within 30 seconds anyway (related Mantis #516) - fixed: process SYNC_CONVERSATIONMODE and save value in ContentParameters (Mantis #527) - fixed: check if shared memory is available before trying to use it - fixed: Impossible to download attachments with iPhone (Mantis #520) - fixed: wbxml tag order for ping should not be relevant (Mantis #490) - fixed: check if folder is known to the device before saving supported fields - fixed: Eurosign (€) gets scrambled when sending email from Windows Mobile (Mantis #456) - fixed: patch to add forgotten licensing exception to LICENSE file (Mantis #462) (thx to Robert Scheck for reporting) - fixed: Streamer and RFC822 class should not be executable by default while z-push-admin should be (Mantis #472) (thx to bencer from Zentyal for reporting) - fixed: typo prevents LOOSE_PROVISIONING (Mantis #476) - fixed: body truncation not always correct (Mantis #467) - fixed: [ZP-7] - SYNC_FILTERTIME_MAX is not always honoured (reported by Vincent aka liverpoolfcfan) - fixed: [ZP-8] - Z-Push's hex2bin() conflicts with php 5.4 hex2bin implementation (reported by Wolfram Quester) - fixed: [ZP-11] - iPad does not understand global status 12 and loops - fixed: [ZP-12] - Status SYNC_STATUS_INVALIDSYNCKEY is not interpreted by Touchdown (reported by Erwin aka ebogaard) - fixed: [ZP-13] - PING request discards old states (reported by Jan aka deHoeninger) - fixed: [ZP-14] - z-push-admin does not work with combined backend (reported by dupondje) - fixed: [ZP-15] - Moving messages from main store to e.g. public folder does not work - fixed: [ZP-16] - StatusException SYNC_STATUS_FOLDERHIERARCHYCHANGED can sometimes not be sent to device - fixed: [ZP-17] - Unable to create appointments on Windows Phone 7 (reported by Rudi aka rwaron) - fixed: [ZP-18] - Z-Push-Admin is not able to display contact's name in case object needs attention - fixed: [ZP-19] - z-push-top does not work on Suse systems - fixed: [ZP-20] - PHP Warning: Undefined variable: smartreply (8) - fixed: [ZP-24] - Broken messages in first batch cause endless loop on iOS - fixed: [ZP-25] - Loop detection works on all messages to be exported - fixed: [ZP-27] - Loopdetection has to consider ids of objects which are streamed - fixed: [ZP-28] - Errors on SMS sync - fixed: [ZP-32] - Android unable to send mail, worked with previous v2.0beta1-1139 (reported by bbs2web) - removed: AlterPing of backends is obsolete (Mantis #585) Zarafa/ICS - implemented: WasteBasket support (deleted emails are moved there) (Mantis #573) - implemented: refactored Zarafa backend - implemented: refactoring of the centralized MAPI access - added: function for defining native body type from plain text, html, rtf and rtfinsync properties - added: set default internet cpid property value to 65001 (utf-8) if the store supports unicode properties - added: PR_ATTACH_CONTENT_ID_W unicode type definition - added: MAPI definitions for the attendee type (relevant to Mantis #565) - added: [ZP-101] - If login fails print value of mapi_last_hresult() - changed: [ZP-49] - Improve the server side search - changed: [ZP-76] - Research and merge phpmapi changes - changed: only set the truncation size data if device set it in request - changed: Authenticated messages not to be logged on INFO level (Mantis #619) - changed: GAL search returns not all values available in ADS (Mantis #469) - changed: do not truncate emails if body preference type is SYNC_BODYPREFERENCE_MIME or the iphone won't be able to show attachments - fixed: typo in settingsUserInformation resulting in warning - fixed: Use constants instead of hard-coded values for the Internet CPID property (Mantis #579) - fixed: Mails deleted on Android phones throw StatusException (Mantis #559) - fixed: Appointments with many attendees can cause high load and are now ignored (Mantis #577) - fixed: The timezone for appointments on some android devices is wrong (Mantis #580) - fixed: Android calendar is not working (Mantis #602) - fixed: Reply/forward flags are not set after answering an email on Android (Mantis #604) - fixed: Errors when deleting a folder on a mobile (Mantis #626) - fixed: Emails sent from mobiles stay in Outbox folder (Mantis #605) - fixed: Tasks and SyncMeetingRequest reminder flag should always be set (Mantis #617) - fixed: iOS 5 tasks' due vs. start date prevent synchronization (Mantis #598) - fixed: Only first of multiple lines of a note's body is saved on the server (Mantis #599) - fixed: Changed notes of appointment not saved on the server (Mantis #601) - fixed: SyncObject->Check() should not set reminder if no reminder is set for appointment (Mantis #569) - fixed: PR_SEARCH_KEY is set to a wrong value (Mantis #570) - fixed: Delete and Move operations are not pushed to the mobile (Mantis #595) - thx deHoeninger for reporting - fixed: a task created on a mobile doesn't appear in Outlook Today view. The due date must be at 12:00AM straight (Mantis #525) - fixed: catch MAPIException in ExportChangesICS() when Exporter can not be configured (Mantis 490) - fixed: Sending attachments from Nokia may result in empty file at recipient (Mantis #451) - fixed: Synchronizing an empty folder an additional ICS exporter is registered on each request (Mantis #480) - fixed: emails sent from an iPhone with iOs 4 are delivered to the recipient with an empty subject (Mantis #529) - fixed: Android 2.2 displays emails as priority low which have NO priority (Mantis #439) - fixed: body truncation not always correct (Mantis #467) - fixed: HTML code in mails sent from Android 2.2 (Mantis #472) - fixed: Lazy conflict detection shows warnings and does not check conflicts (Mantis #481) - fixed: Appointment is not synchronized after meeting invitation was accepted in the Webaccess of ZCP 7 (Mantis #475) - fixed: Saving empty categories fails (Mantis #468) - fixed: Contacts Birthday not properly saved from Android (Mantis #487) - fixed: Contacts created on Android phones present doubled names (Mantis #486) - fixed: Meeting request responses can not be processed correctly (Mantis #485) - fixed: GAL search shows 'undefined offset of PR_ACCOUNT' warnings (Mantis #488) - fixed: Exception body changes of recurring appointments are not displayed or saved on mobile (Mantis #461) - fixed: [ZP-4] - Double quoted email addresses show invalid high character encoding on iOS - fixed: [ZP-6] - Setting reminder to none does not work - fixed: [ZP-34] - Warning when saving contact without zip code or state - fixed: [ZP-61] - Inline.txt instead of inline image on Android - fixed: [ZP-83] - Email sent from a mobile with ZCP 7.0.6 or higher is not moved to Sent Items folder - fixed: [ZP-84] - No recipient for emails in Sent Items folder on the mobile - fixed: [ZP-85] - Impossible to send an email using HTC one X (Undelivered Mail Returned to Sender) - fixed: [ZP-86] - Unable to respond to a meeting request on Android - fixed: [ZP-33] - RIM's Playbook and HTC one X only display some calendar entries - fixed: [ZP-58] - WBXMLException when replying an email - fixed: [ZP-60] - Zarafa-fsck reports invalid property on recurrent calendar items created on iPhone - fixed: [ZP-66] - Apple Iphone shows hidden maps - fixed: [ZP-89] - Not able to create new map in parent tree on Iphone - fixed: [ZP-92] - z-push-admin shows notices - fixed: [ZP-97] - No recipient for emails in Sent Items folder on the mobile - fixed: [ZP-100] - Warning when synchronized folders entryid can not be opened - fixed: [ZP-112] - Process hangs consuming 100% CPU on performing GetEmail()->processMeetingCancellation() - fixed: [ZP-113] - Print php-mapi version into log - fixed: [ZP-114] - Moves do not trigger notifications on source and destination folder if both are monitored in a changes sink - fixed: [ZP-117] - Warning when MeetingRequest can not be opened - fixed: [ZP-119] - Replied/forwarded message does not contain the original message - fixed: [ZP-120] - Range in the GAL search response is invalid - fixed: [ZP-125] - Fatal WBXML on recipient search in the GAL - fixed: [ZP-127] - PHP Fatal error: Call-time pass-by-reference has been removed with PHP 5.4 - fixed: [ZP-128] - Heartbeat export optimizations disable loop detection - fixed: [ZP-135] - Special chars are broken on an Android 4 and Windows Phone Diff Backend - implemented: refactored Diff backend - changed: [ZP-98] - Remove the constructor from BackendDiff class IMAP backend - implemented: imap folder ids are masked as hex ids to be used by the processor. Incoming hex ids are unmasked on the fly - implemented: ChangesSink functionality previously provided by AlterPing() - fixed: importance status set by imap backend was incompatible with AS-importance - fixed: no attachments on the mobile if Content-Disposition header is not set (Mantis #458) - fixed: email is not quoted-printable encoded even if the header states it (Mantis #460) - fixed: email can't be synchronized with certain date format in header (Mantis #454) - fixed: Folders with special chars break synchronization (Mantis #463) - fixed: Iconv conversion from UTF7 does not work (Mantis #574) - fixed: [ZP-2] - Typo in DiffState prevents conflict detection