01-02-2010, 03:41 PM
[attachment=1644]
Abstract
The popularity of mobile computing and communications devices can be traced to their ability to deliver information to users when needed. Users want ubiquitous access to information and applications from the device at hand, plus they want to access and update this information on the fly.
The ability to use applications and information on one mobile device, then to synchronize any updates with the applications and information back at the office, or on the network, is key to the utility and popularity of this pervasive, disconnected way of computing. Unfortunately, we cannot achieve these dual visions: Networked data that support synchronization with any mobile device Mobile devices that support synchronization with any networked data Rather, there is a proliferation of different, proprietary data synchronization protocols for mobile devices. Each of these protocols is only available for selected transports, implemented on a selected subset of devices, and able to access a small set of net-worked data. The absence of a single synchronization standard poses many problems for end users, device
manufacturers, application developers, and service providers. synchronization protocol that can be used industry-wide.
Contents
Introduction
Need for a Common Data Synchronization Protocol
Common Synchronization Protocol
Characteristics of a Common Synchronization Protocol
The SyncML Initiative
SyncML Protocols
How SyncML Works
Types of Synchronization
Synchronization Procedure
Conclusion
References
Introduction
The popularity of mobile computing and communication devices can be traced to their ability to deliver information to users when needed. Users want ubiquitous access to information and applications from the device at hand, plus they want to access and update this information on the fly.
A long-standing obstacle to the advancement of ubiquitous computing has been the lack of a generalized synchronization protocol. Until recently, the available synchronization protocols were proprietary, vendor-specific, and supported synchronization only on selected transports, implemented on a selected subset of devices, and able to access a small set of net-worked data. This has slowed development in the area of mobile computing and been a common source of frustration for users, device manufacturers, service providers, and application developers. SyncML is a new industry initiative to develop and promote a single, common data synchronization protocol that can be used industry-wide. Driving the initiative are Ericsson, IBM, Lotus, Motorola, Nokia, among others. SyncML is intended as a common language that enables smooth, efficient synchronization of personal and business information over fixed or mobile networks. Its aim is to facilitate the synchronization of networked information with various devices running SyncML-compatible applications. As the first universal synchronization protocol, SyncML offers true freedom to users of mobile devices by allowing them to send and receive up-to-date information between their mobile applications and their office or home-based systems regardless of the platform, manufacturer, or application. SyncML leverages Extensible Markup Language (XML), making SyncML a truly future-proof platform.
Need for a Common Data Synchronization Protocol
All the popular mobile devices - handheld computers, mobile phones, pagers, laptops - synchronize their data with network applications, desktop calendars, and other locations where information is stored. This ability to access and update information on the fly is key to the pervasive nature of mobile computing. Yet, until recently, almost every device uses a different technology for performing data synchronization. Trends in the computing industry are moving us toward a future in which users expect almost any kind of tangible device to be networked. While we haven't reached that state yet, we already have access to a lot of computing devices that can be carried around and that possess sufficient computing power to log on to networks: personal digital assistants, mobile phones, laptops, etc. And the devices are only getting smaller: Current industry trials are testing the feasibility of wearable computers. Users who own these devices expect to be able to access networked data from almost any place and in almost any situation. And since these users would find a physical, wired connection to a network impossibly cumbersome, in all probability they will access network data wirelessly. This is what is popularly known as pervasive computing (PvC). Pervasive computing networks are huge distributed systems whose characteristics differ slightly from traditional distributed systems.
The Synchronization Problem
The industry offers a variety of non-interoperable data synchronization products, each connecting data from a few types of data repositories to a few devices. Each protocol functions only for selected transports, and is implemented on a few devices. Most synchronization products use different communication protocols over the network. The proliferation of non-interoperable synchronization technologies complicates the tasks of users, device manufacturers, service providers, and application developers. The lack of a common data synchronization protocol is impeding the growth in use of mobile devices, restricting userâ„¢s ability to access data, and limiting the delivery of mobile data services.
Common Data Synchronization Protocol hat is a Synchronization Protocol
Mobile users are not always connected to a network and its stored data. Users retrieve data from the network and store it on the mobile device, where they access and manipulate the local copy of the data. Periodically, users reconnect with the network to send any local changes back to the networked data repository. Users also have the opportunity to learn about updates made to the networked data while the device was disconnected. Occasionally, they need to resolve conflicts among the updates made to the networked data. This reconciliation operation - where updates are exchanged and conflicts are resolved - is known as data synchronization. Data synchronization, then, is the process of making two sets of data look identical. For a mobile device, synchronization applies to the data that the mobile device stores locally. A data synchronization protocol defines the workflow for communication during a data synchronization session when the mobile device is connected to the network. The protocol must support naming and identification of records, common protocol commands to synchronize local and network data, and it can support identification and resolution of synchronization conflicts.
Benefits of a Common Synchronization Protocol
Let's look at each group that will benefit from adoption of an industry-wide data synchronization protocol.
End Users - The user of mobile devices is probably using a different synchronization product with every device. That is, there is one procedure for synchronizing files between a laptop and networked data, another for synchronizing calendar on a handheld computer, and yet another remote access to email. Each technology can synchronize only a few applications, or is limited to a particular type of network connection. This arrangement is expensive to install, confusing to configure and operate, and costly to administer. With SyncML, they will be able to buy devices that synchronize with a broader range of data.
Device Manufacturers - While every device manufacturer wants to support the technologies that will support the data access needs of all users and service providers, in practice a device will support one data synchronization technology. This choice is forced upon the manufacturer by constraints of storage space, memory, power consumption, and cost. Device manufacturers will benefit from a common protocol that will make the device interoperable with a broader range of applications, services, and network and transmission technologies.
Service Providers - Service providers moving into the growth arena of application hosting are particularly concerned that a proliferation of synchronization technologies will make it impossible to deploy and support their customers in a cost-effective manner. Already, to support the range of data types and devices in use, service providers must install and configure multiple server infrastructures, maintain and support that infrastructure, and maintain compatibility and performance. The alternative now available, to use a single solution for data connectivity, involves the risk of a tight coupling to a proprietary solution. With SyncML, they will be able to provide connectivity to a wider selection of applications.
Application Developers - Choosing to support multiple synchronization technologies enables an application to support more types of devices and networked data, but that choice comes at a cost. The developer loses the flexibility of evolving the choice of networked data repository while maintaining backward compatibility. It also increases the cost of application development and the complexity of the resulting product. The added complexity of the networked data repository can create a barrier to installation and adoption by service providers.
Characteristics of a Common Synchronization Protocol
The goal of a common synchronization protocol is symmetric. It would connect any to any, over any network. That is, it would: Synchronize networked data with any mobile device Synchronize a mobile device with any networked data The data synchronization protocol would synchronize networked data with many different devices, including handheld computers, mobile phones, automotive computers, and desktop PCs. A user could access and manipulate the same set of data from different devices. For example, a user could read e-mail from either a handheld or a mobile phone, and still maintain a consistent, updated record of which messages had been read.
Similarly, with any-to-any synchronization, mobile devices could support more types of data, including e-mail, calendar, contact management information, enterprise data stored in databases, and documents on the web. With such functionality a user who received an order via e-mail could access the company inventory system on the same device to determine a delivery date. To accomplish this goal, the protocol needs the following characteristics: Operate effectively over wireless and wired networks
Support a variety of transport protocols Support arbitrary networked data
Enable data access from a variety of applications Address the resource limitations of the mobile device Build upon existing Internet and Web technologies The protocol's minimal function needs to deliver the most commonly required synchronization capability across the entire range of devices.
Effective over Wireless and Wired Networks
A common synchronization protocol must work over all networks used by mobile devices, both wireless and wired. The various wireless networks commonly used by mobile devices demand the most from a protocol, since these wireless networks share common features of high network latency, limited bandwidth, relatively high packet costs, and low reliability of both data and connectivity. High network latency - Network latency is the delay introduced when a packet is momentarily stored, analyzed and then forwarded. Wireless networks, with a high latency, require a robust synchronization protocol. Limited bandwidth - To minimize bandwidth requirements and the associated processing demands, the protocol should allow alternate binary encoding techniques to both the data and the synchronization commands. The WBXML (WAP Binary XML) standard adopted by the WAP Forum and submitted to the W3C represents a good candidate encoding technique for limited bandwidth environments. Relatively high packet costs - The protocol must minimize the number of request response interactions between the device and the networked data. An optimal protocol would generate a single request-response pair of messages. In this model, a request from a mobile device would contain all updates to its local data. The response message would provide the updates, with conflicts already identified and possibly resolved. Any protocol adopted by the industry should seek to enable this minimalist messaging model. Low reliability of both data and connectivity - In order to function with only intermittent connection to the network, the protocol must survive inadvertent disconnection during the synchronization procedure. Even when a disconnection is encountered, the protocol must ensure that the device and the networked data repository stay consistent and make progress when the connection is reestablished.
Support Various Transport Protocols and Media
Since wireless networks employ different transport protocols and media, the protocol must work smoothly and efficiently over: HTTP (i.e. the Internet)
WSP (the Wireless Session Protocol, part of the WAP protocol suite)
OBEX (i.e. Bluetooth, IrDA, and other local connectivity) Pure TCP/IP networks Proprietary wireless communication protocols An effective synchronization protocol cannot depend on any capabilities that cannot be made available over these transports. To be efficient, the protocol should minimize duplicating features provided by the transport. A reliable request-response model can be efficiently deployed across all of these transport protocols. An effective protocol could be built on this model. Moreover, defining a MIME (Multi-Purpose Internet Mail Extensions) content-type for the protocol units will allow the protocol to be transported across any of these different transports. Optional enhancements to the transport protocol should include security and compression capabilities.
Support Arbitrary Networked Data
Since a design goal is for mobile devices to communicate with a broad range of networked data, the protocol must support concurrent synchronization with multiple network data repositories. The protocol should not mandate how data is represented or structured on the device or within the networked data repository after synchronization is complete. To ensure interoperability, the protocol must describe how common data formats are represented over the network. To ensure extensibility, the protocol should permit the definition of new data formats as needs arise. In addition, the protocol must allow implementers to define experimental, non-standard synchronization protocol primitives. The common data formats that a protocol must support from the outset include: Common personal data formats, such as vCard for contact information, vCalendar and iCalendar for calendar Collaborative objects such as e-mail and network news Relational data XML (the Extensible Markup Language) and HTML documents Binary data, binary large objects, or "blobs"
Enable Data Access from a Variety of Applications
The specification shall be programming language independent. In order to work effectively with many applications, the synchronization protocol must not make assumptions about the programming language and cannot assume that both ends of the synchronization process share a single language environment. To facilitate rapid deployment of the protocol the reference code shall be provided for a common programming language. However, this will not restrict any implementation to only this language.
Address the Resource Limitations of the Mobile Device
Mobile devices have limited memory and processor capacity, a characteristic that sets the rules for developing a synchronization protocol.
The protocol implementation needs to fit within the memory footprint of the common mobile devices on the market today, in either static code or run-time execution space. In addition, the data exchanged by the protocol itself should be small and require minimal code to transfer it to and from the device. Data exchanged using the protocol may be encoded in a binary format (such as WBXML) to reduce memory requirements for storing received synchronization messages and reduce processor resources needed to parse and process that data. A small footprint and processor requirement gives the protocol several key advantages. A small code size eases the porting overhead and increases the likelihood that implementations will be available for all processor and/or operating system platforms. It also reduces device manufacturers' cost of adoption, an important consideration.
Build Upon Existing Internet and Web Technologies
Where possible, the protocol needs to make use of existing Internet and Web technologies. These technologies are implemented widely and are well tested, so their use within the protocol will ensure easy implementation and interoperability testing. XML, the Extensible Markup Language, is rapidly emerging as the lingua franca for representing structured data over the Web. To the extent possible, the protocol should use XML to represent data being exchanged during a synchronization session. Other useful standards in this space include MIME for registering the format of the data synchronization protocol messages.
Promote Interoperability
The protocol needs to be interoperable, synchronizing networked data with any mobile device and synchronizing a mobile device with any networked data. This includes a market space where mobile devices are small-footprint devices with minimal processor and memory resources and powerful network servers capable of executing sophisticated synchronization logic.
The SyncML Initiative
To accelerate the market's vision of ubiquitous data access from any device to any networked data, Ericsson, IBM, Lotus, Motorola, Nokia, Palm Inc., Psion, Starfish Software has formed the SyncML initiative. This industry initiative will deliver a common synchronization protocol that can be used industry-wide. SyncML initiative will work with end users, device manufacturers, data providers, infrastructure developers, application developers, and service providers to define a common mobile data synchronization protocol, SyncML. This protocol will meet the resource constraints of mobile devices and wireless networks and will provide the extensibility to support a range of data types. The goal of the SyncML initiative is to deliver the protocol in the future for formal adoption and maintenance by an established standards body. To enable adoption of the SyncML, SyncML initiative will deliver: An architectural specification
Two protocol specifications (SyncML representation protocol and SyncML synchronization protocol) Bindings to common transport protocols
Interfaces for a common programming language An openly available prototype implementation of the protocol
SyncML Protocols
The SyncML programming framework is based on two protocols: SyncML Representation protocol and SyncML Synchronization protocol. The SyncML Representation protocol defines the representation format of SyncML messages (in XML) and details the inner workings of the SyncML framework. The SyncML Synchronization protocol defines the actions between a SyncML client and a SyncML server. In order to build a SyncML-compliant product, you must adhere to the requirements of these two protocols. The important components of the SyncML protocol are:
Change log
Being a synchronization protocol, SyncML requires that both client and server maintain information about changes or modifications (i.e., replacement, addition, or deletion of data) to the data in their databases. The mechanism by which this information is tracked is called a change log. SyncML does not specify the format of the change log; however, upon synchronization each device must be able to specify the data items that have changed since the previous synchronization.
Map operation
SyncML is based on the principle that the client and the server may each have their own unique identifiers (IDs) for items in their databases. The client ID is known as a locally unique identifier (LUID); the server ID is known as a globally unique identifier (GUID). These IDs may or may not be congruent between client and server. Because the IDs can be different, the server must maintain an ID mapping table for all items exchanged between itself and the client. That is, the server must know which client IDs and which server IDs point to the same data item. LUIDs are always assigned by the client device. This means that even if the server adds an item to the client device, the client will assign the LUID for that item. The client will then use the Map operation to send the LUID of the new item to the server. Following the Map operation, the server will update its mapping table with the client LUID.
Sync anchors
The two sync anchors, Last and Next, are always sent when a synchronization session is initialized. The Last sync anchor describes the last synchronization event from the point of the sending device; the Next sync anchor describes the current event of synchronization from the point of the sending device. In this way, the client and the server send sync anchors to each other. Upon receiving a Next sync anchor, the receiving device must store it until the next synchronization. Upon the next synchronization, the receiving device will compare its stored sync anchors with the sending device's Last sync anchor, to determine whether any failures have occurred in the sending device's data. If the Last and Next sync anchors match, the receiving device concludes that no failures have occurred. If they don't match, the device can request an appropriate action from the other device, such as a slow sync. The stored sync anchor is updated only when the session has ended successfully.
Conflict resolution
A version conflict generally arises when a single data item is modified on both the client and the server database, resulting in two different versions of the same data. A synchronization protocol must have a policy for resolving such conflicts. In the SyncML, conflict resolution is a function of the sync engine. Generally, the SyncML server's sync engine resolves version conflicts; it is also possible for the SyncML client's engine to provide some of this functionality. The SyncML Representation Protocol provides both the functionality to notify the SyncML client of a resolved conflict and the status codes for common resolution policies. If the server's sync engine resolves a conflict, it will use the status codes and notification
functionality to notify the client of the conflict and define the terms of its resolution.
Security
SyncML is intended to provide a framework for secure data synchronization. SyncML itself does not define any new security schemes. Instead, it provides the framework to challenge authentication and authorization at different layers of the network. SyncML defines a user authentication mechanism for three different levels of the protocol : the server level, the database level, and the object level. SyncML mandates only that its authentication mechanism be supported at the server level. To be compliant with the SyncML protocol, a SyncML client or server must support both basic and MD5 authentication. Authentication at the database and object levels is optional.
How SyncML Works
The SyncML representation protocol is defined by a set of well-defined messages (XML documents or MIME) that are shared between synchronizing devices. It supports data synchronization models that are based upon a request/response command structure, or those based upon a "blind push" structure. The SyncML representation protocol specifies what the result of various synchronization operations should be, based upon a synchronization framework and format that accommodates different data synchronization models.
SyncML Representation Protocol version 1.0.1
In the above figure, Application A represents a networked service that provides synchronization with other client applications (Application B). The server and client are connected over any network transport (HTTP, WSP). The client uses the Sync Client Agent to access the network and send messages to the server via the SyncML Adapter and SyncML Interface (SyncML I/F). The server, or Application A, through the Sync Server Agent, receives or sends messages, and manages the entire sync process through the Sync Engine. A SyncML I/F is merely an API to the SyncML Adapter.
SyncML operations are conceptually bound into a SyncML Package, which is a conceptual frame for one or more required SyncML messages. A SyncML message is a well-formed XML document identified by the SyncML root or document element type. The document consists of a header (SyncHdr element type) and a body (SyncBody element type). The header specifies routing and versioning information, while the body is a container for one or more SyncML Commands. The commands are containers for other element types that describe the specifics of the command, including any synchronization data or meta information. Incorporated here, too, are features such as SyncML Data Formats (a common set of media types for commonly accepted information such as calendars and contacts) and SyncML Capabilities Exchange (in which a SyncML client and server determine what device, user, and application features each supports) are incorporated. For example, a mobile phone acts as the SyncML client, and a server acts as the SyncML server. The SyncML client sends a message to the SyncML server regarding changes to data made on the client. The server then synchronizes the data within the SyncML messages with data stored on the server, and returns modifications back to the SyncML client. The SyncML client contains a sync client agent, and typically has the role of sending modifications first to the server. The client is typically a mobile phone or PDA, and must also be capable of receiving messages back from the server. The SyncML server contains the sync server agent and the sync engine, and usually waits for the client to initiate synchronization, although the server can initiate synchronizations if unsolicited commands are supported on the transport protocol level.
Types of Synchronization
SyncML specifies seven different synchronization types, as follows:
1) Two-way sync: Two-way sync is a normal synchronization type in which client and server exchange information about any modifications to the data each contains. The client always initiates this exchange by sending a request to the server. The server processes the synchronization request and the data from the client is compared and unified with the data on the server. After that, the server sends its modified data to the client device, which is then able to update its database with the data from the server. Once it has updated its database, the client sends back the server all the necessary mapping information.
2) Slow sync: The slow sync is a form of the two-way synchronization in which all the items in the client databases are compared with all the items in the server databases on a fieldby- field basis. A slow sync can be requested if the client and server sync anchors are mismatched or if the client or server loses its change log information. In practice, the slow sync means that the client sends all its data to the server and the server does a fieldby- field analysis, comparing its own data with that sent by the client. After the sync analysis, the server returns all the modification information to the client. In turn, the client returns the mapping information for all data items added by the server.
3) One-way sync from client only: This is a synchronization type in which the client sends its modifications to the server but the server does not send its modifications back to the client.
4) Refresh sync from client only: Here the client exports all its data from a database to the server. The server is expected to replace all data in the target database with the data sent by the client.
5) One-way sync from server only: With this, the client gets all modifications from the server but the client does not send its modifications to the server.
6) Refresh sync from server only: In this scenario the server exports all its data from a database to the client. The client is expected to replace all data in the target database with the data sent by the server.
7) Server-alerted sync: Here the server informs the client of the need to initiate a specific type of synchronization with the server.
Synchronization Procedure
With this basic understanding of SyncML, we can begin to talk in detail about what happens between a SyncML client and the server during a synchronization procedure. Because twoway synching is the most common synchronization procedure, we'll use that procedure as our example.
A two-way synchronization procedure can be broken out into the following broad steps: 1) Sync initialization 2) Two-way synchronization 3) Data mapping
Sync initialization
No synchronization procedure can occur until the client and server have exchanged initialization packages. In an initialization procedure, the server and client devices generally exchange the following information:
Service and device capabilities Requested databases access, type of sync desired Authentication information Sync anchors are also exchanged as a part of the initialization procedure. Figure below illustrates the basic steps of a SyncML initialization procedure.
SyncML initialization
Upon initialization, the client sends the server an initialization package (Pkg #1). With this package, it must inform the server which databases it wants to synchronize with and which types of synchronization it desires (e.g., two-way sync, slow sync, etc.). The client indicates the databases it wants to synchronize with by using a series of alert commands. Alert commands are specifically used to convey notifications and requests. So, for example, a mobile device would use an alert command to request a "client-initiated, two-way synchronization" with a network server. For each database the client wants to synchronize with, it must send a separate alert command. It is optional for the client to send its authentication information in the first package of the initialization. If the client does not send this information with the first initialization package, the server will initiate an authentication procedure by sending the client a challenge command. At that point the client must be able to provide authentication information.
When the server has received the first initialization package from the client, it completes the initialization phase by sending its own initialization package (or status update) to the client (Pkg #2). To prepare this package, the server checks the status of all the databases the client wants to synchronize with, compares the device capabilities of the client with its own, and matches the sync anchors for their respective databases. It then sends the client an initialization package that includes the alert and related status elements for each database the client wants to synchronize with. To complete the sync initialization, the client must respond to the commands (e.g., Alert, Put, and Get) sent by the server. The status elements and result element associated with each alert command can be returned in the first package of the actual synchronization (Pkg #3).
Two-way synchronization
Two-way synchronization is a normal synchronization type in which the client and the server exchange modifications to the data each contains. The client always initiates this exchange by sending its data modifications to the server. This is called a synchronization request. The server processes the synchronization request, comparing and unifying the data from the client with the data in its databases. After that, the server sends its modified data to the client, which is then able to update its database with the data from the server. Figure below illustrates the basic steps of two-way synchronization.
The client always initiates a two-way sync, by sending its modifications to the server (Pkg #3). The client uses internal change log information to determine all the modifications that have occurred in its databases since the last synchronization. You will recall that modifications include the addition of new entities, deletion of previous entities, or updates to existing entities. Each change type is sent to the server using a different SyncML command. For example, all additions are sent using the Add command and all deletions are sent using the Delete command.
On receiving the sync package from the client, the server processes the commands sent by the client, incorporating all the modifications into its databases. The server is responsible for handling any version conflicts that may arise in this process. The server then builds a status report consisting of all the commands sent by the client combined with its own data modifications and sends its synchronization package to the client (Pkg 4).
Data mapping
Upon receiving the synchronization package from the server, the client processes all the commands. It then creates a data status update to inform the server of the results of the data update on the client side (Pkg #5). This status update will include a set of Map operations, which allow the server to match up all the LUIDs and GUIDs in its mapping table. The update package may not be sent if the server has indicated that it does not require a response to its last package. If the client does not send the update, it must cache the Map operations until the next synchronization.
Conclusion
SyncML, launched in February 2000, marked the coming together of industry leaders (Ericsson, IBM, Motorola, and Nokia, among others) to resolve the synchronization problem. With the February 2002 release of the SyncML version 1.1 specification, we have our chance to begin working in earnest with what already promises to be a groundbreaking protocol.
SyncML is an ideal choice for mobile computing, freeing up memory for other applications. Nokia was the first company in the world to introduce SyncML-enabled products. SyncML an open industry specification for data synchronization is designed to meet the needs of the mobile user and their any-device, any-network synchronization needs. With SyncML, networked information can be synchronized with any mobile device, and mobile information can be synchronized with any networked applications. SyncML is not limited to mobile phone users - it also enables synchronization over other transmission channels such as fixed networks, infrared, cable, or Bluetooth. In fact, SyncML will even allow users to synchronize information stored on compatible Web-based applications.
References
1) "Keynote Speech: Introducing SyncML and Data Synchronization"
Douglas Heintzman, Chairman of the SyncML Initiative
2) A beginner's look at the SyncML protocol and procedures
Apr 2002, Chandandeep Pabla
3) An introduction to SyncML Oct 2001, Scott Stemberger
4) http://openmobilealliancetech/affiliates/whitepaper.pdf
5) http://www-106.ibmdeveloperworks/wireles...i-syncml2/
syncmlintensive.html
6) http://nokia.co.uk/nokia/0,8764,18359,00.html
7) http://cellular.co.za/syncml.htm
