Why doesn’t SyncML synchronization works?

October 29, 2009

The short answer

This is most often because:

  1. You haven’t supplied the proper database names (spelling mistake or just didn’t change the default ones)
  2. You haven’t supplied the proper user name and passwords (the later is case sensitive)
  3. There is a bug in your device/program or in picoBeat SyncML module. Or possibly both (ugh!).

The long answer

There’s a lot to say about the standards developed by the telecom industry. Somehow, while a cell-phone is just a tiny computer, the telecom guys seem to live on a different planet than the computer guys, and they seem to insist on doing things “their way”.

When it was decided to bring the Internet to cell-phones, HTML would have been an obvious choice. But no, a different standard was built from scratch: WML. WML 1.0 wasn’t really a big success, for many reasons including the fact that Web developers do not like having to learn a whole new markup language, so WML 2.0 tried to correct this by being some limited form of HTML. Similarly, pictures were expected to be stored in a new WBMP format, rather than one of the many existing image format that already exists and are widely supported.

This bring us to SyncML: it’s a synchronization standard by the telecom industry. Two good things about it: it’s built with XML and it’s pretty flexible. Two bad things about it: XML is encoded in a new custom format called WBXML, somewhat negating the advantage of using XML, and it’s way too flexible. This comes from contradicting constrains: making the standard as flexible and open-ended as possible, but making it work on underpowered devices such as entry-level cell phones. As such the standard requires devices to accept informations in many different ways and to be very flexible in exchanging data, which requires complicated programming and a lot of processing power. But at the same time it allows the phone to signal that it has little memory capacity and bandwith. Why would a device capable of processing a complex XML-based protocol be only capable of storing a handful of appointments or processing more than 2,5 Kb at a time is beyond my understanding.

This flexibility is also the reason there’s no standard name for databases (tasks, appointments, contacts, etc.)  in SyncML and why users have to type them in when configuring their cell-phone. This complexity also means a lot of cell-phones have bugs – some of them so bad as to reboot when receiving certain standard-compliant SyncML data. Sadly there are dozens of phone manufacturers and thousands of models out there and it’s impossible to properly test every one of them with picoBeat, so I know for a fact that some phones just won’t work. Things will improve over time and hopefully most devices will work.

19 Comments »

  1. Hello, i’m trying to put PB to work in my nokia 5800XM, but it keeps saying ‘wrong username or password’. I triple-checked my configs and it’s all ok, but it keeps giving me that error…
    Are there any confs I should put in ‘network name’ and ‘network passwd’ besides my own login/passwd ? What is the server port? 80? does the contacts/notes/etc need to be typed like ./notes in the proper fields?
    I tried it all, but no lucky… Could you help?

    thanks!

    Comment by Kebs — November 3, 2009 @ 9:16 pm

  2. Thanks for the feedback! I had a look at the logs and it seems the Nokia 5800 has a little “quirk” in the way it sends the identification infos. I made a small change that should make it work now. You don’t need to use a special port and the database names are as given on the SyncML configuration infos.

    Comment by Pierre — November 4, 2009 @ 4:13 am

  3. Hi Pierre, thanks for the fast answer! I tried with a 5800XM and a N81, and they give me a similar message, \"invalid remote database\" (5800) and \"can\’t open database\" (N81). I\’m using the confs available at the page you mentioned.

    just for the sake of doublechecking, here are my other confs:

    server version: 1.2
    server id: picobeat (gives the same message if I leave it blank)
    host address: http://sync.picobeat.com/
    port: 80
    username and password is the same as the site
    network authentication: no
    contacts remote database: contacts
    sync type: two-way
    Calendar rem. database: cal
    sync type: two-way

    (didn\’t set the other databases yet)

    thanks!

    Comment by Kebs — November 5, 2009 @ 10:17 pm

  4. I checked again, it seems this phone always add “./” before the database name. This is the first time I see this but it actually seems valid according to the specs. I adjusted the server code and it should go OK now (cross fingers). Thanks for your patience!

    Comment by Pierre — November 6, 2009 @ 4:00 am

  5. hi pierre, i´m glad to help! I ´d thank you for your will to help me! I tested the system again and here´ s what I got:

    N81 now can sync contacts, but it didn´t send anything to the server, even giving an ´ok´ message. If I try to sync calendar, it gives a ´system error´ msg.
    The 5800 can´t sync cal neither contacts: it logs in and gives a ´ connection error´. Forsome unknown reason for me, n81changed the server id to alpha.picobeat.com… I tried to change by hand this setting on 5800 but had no luck.
    I tried the database set. with and without ./ ( just in case you look at the logs and finds both tries)
    I´m sorry to give that ammount of trouble… Hope we can get it fixed. Let me know if there´ s something I can do to help you, ok?
    Thanks a lot again!

    Comment by kebs — November 7, 2009 @ 2:27 am

  6. OK it seems both phones use the Nokia S60 software. I’ve tried working with a S60 emulator and found that the “server id” parameter is actually a mandatory information to enter, and the synchronization will fail if it doesn’t match the one provided by the server. You should use alpha.picobeat.com (all the phones I had tried until now didn’t even let you input this parameter and didn’t care).

    I’ve tweaked a few other things and managed to synchronize contacts between the emulator and the picoBeat server. Let me know how it goes now for you.

    Comment by Pierre — November 7, 2009 @ 1:23 pm

  7. Hi Pierre, sorry i couldn’t reply earlier.

    I tried with the N81 and it works like a charm now!! thanks a lot!!

    I will try with the 5800 to see how it goes. 5800 uses a different version of s60 (n81 uses v3 and 5800 uses v5).

    I’m from Brazil and we use a lot of accentuated characters in portuguese… What happens is that in Calendar, the accetuanted chars looks like : “Consci=C3=AAncia=20negra.” for “Consciência negra.”. Note that even the spaces are being replaced by “=20″ codes.

    In Contacts it’s all fine.

    thanks again!

    Comment by Kebs — November 12, 2009 @ 3:24 am

  8. I fixed the problem with accentuated chars, it should be OK now :)

    Comment by Pierre — November 12, 2009 @ 6:51 am

  9. Hi!
    Recently, I started to use picoBeat, and I’m quite amazed. However, I found some problems as well. I’m using a Nokia e52 (s60 v3) phone, and the notes sync is not working, I get a system error message. If I uncheck the notes, it synchronises without errors.

    One more thing: my phone doesn’t sync my tasks. It doesn’t even ask for a “tasks” database. But if I type “tasks” in the calendar database field, It works. So it seems, that my phone is trying to sync the tasks and the calendar events together.

    Comment by RiBBiT — November 29, 2009 @ 7:04 pm

  10. Hi RiBBiT! Could you try a synchronization with only notes, so we can go through the logs and see where it breaks down and find a fix?

    You are right with the calendar and tasks: a few phones actually consider the tasks list to belong to the calendar database and will only accept to sync both events and tasks at once. Synchronization should work with picoBeat but only with the calendar OR the tasks (depending if you sync with “cal” or “tasks”) but not both. We plan on building a solution for this situation but there are a few difficulties involved so it might take a few days/weeks.

    Comment by Pierre — November 30, 2009 @ 4:49 am

  11. Hi Pierre!

    Thank you for your reply. I tried it with only the notes, and the error message was exactly the same: \"System error\".

    Comment by RiBBiT — December 2, 2009 @ 2:03 pm

  12. Thanks for testing! The logs don’t really show anything wrong – no error is reported by either the phone or the server, so it’s not clear what exactly goes wrong. There was however a minor bug related to notes that has been fixed, so maybe this will work better now. Let me know if this helps.

    Comment by Pierre — December 2, 2009 @ 5:34 pm

  13. I tried again. Unfortunately it still doesn’t work.

    Comment by RiBBiT — December 3, 2009 @ 4:39 pm

  14. OK, I found and corrected another tiny notes-related issue as well, hopefully this time this will work.

    Comment by Pierre — December 3, 2009 @ 4:43 pm

  15. I’m still getting this meaningless “System error” message with the notes. It must be a bug in the software of my phone.

    Comment by RiBBiT — December 7, 2009 @ 7:50 am

  16. I tweaked another little thing just in case – it’s quite insignificant but it can only help. I ran some test with the Nokia S60 emulator and notes synchronization worked right away, so maybe that tweak helped or maybe it’s really a specific problem of the E52.

    As a side note there’s good progress made on the combined tasks+events calendar synchronization, so hopefully this week it won’t be a problem anymore.

    Comment by Pierre — December 7, 2009 @ 10:01 am

  17. Now, It works perfectly. Thank you very much!

    Comment by RiBBiT — December 7, 2009 @ 1:33 pm

  18. I am testing picobeat. I found that when the server sends contact data to client. The vCard data has 2 flaws:
    1. The address field does not conform to vCard standard. vCard uses ; to separate 7 fields of address. The one sent by Picobeat server has the first 2 fields missing. Both SyncML Client for Open Contacts and Funambol plug-in for Outlook could not then read the address field.
    2. Using CDATA to store data has flaws: if you have a ]]> in a vCard record, then XML will have to have 2 CDATA blocks to escape.

    The following is the vCard data sent by Picobeat, though the server could accept the correct vCard data from the client. This mean: the parser is OK, while the writer has flaws.

    <![CDATA[BEGIN:VCARD
    VERSION:2.1
    REV:20091216T123814Z
    FN:Contact Of Outlook
    N:Outlook;Contact;Of;;Sr.
    BDAY:1978-10-07
    ADR;HOME:32 Kagaga Ave;Home city;VIC;3000;Australia
    EMAIL;INTERNET:mrmsoutlook@earth.com
    URL;HOME:http://www.somewhere.com
    URL;WORK:http://www.somewhere.com
    ORG:General Business;this dept
    TITLE:Mr.
    NOTE:An example with a lot of fields of a MS Outlook contact imported.
    UID:8487.contacts@alpha.picobeat.com
    TEL;WORK;VOICE:7997898
    TEL;WORK;FAX:88888888
    TEL;HOME;VOICE;PREF:1111111
    TEL;HOME;FAX:(2) 2234-4545
    TEL;CELL;PREF:999999
    END:VCARD
    ]]>

    Comment by Andy — December 16, 2009 @ 8:51 am

  19. Thanks Andy, there was indeed a bug with the vCard export function that is fixed now.

    Regarding the CDATA: vCard requires end-of-line CRLF and also uses specific spacings for line folding. As different XML parsers and encoders treat those white-spaces differently, it is safer to enclose the whole vCard in a CDATA section. It seems that’s what most SyncML clients also do so that is probably the way they expect to receive their data from the server. In the rare event you have a ]]> in the vCard content, using two CDATA blocks shouldn’t break a proper XML parser.

    Comment by Pierre — December 16, 2009 @ 10:55 am

Follow any responses to this entry through the RSS feed. You can comment or trackBack from your own site.

Leave a comment

Security Code: