iCal and Google Calendar synchronization

Sunday, December 15, 2024 • ical calendar

This post is a about a small detail I learned while investigating, why the changes in my (self written) ical calendar export didn’t synchronize to Google calendar. In other words, the changes to an event, e.g. the updated start/end times or repeating options didn’t show up on Google.

The problem

So, caching definitely plays a role in that. Google itself states that changes to the imported ical calendar don’t show up immediately, but take up to 12 hours. Actually, I only find this statement in a community post Intergrated iCal calendar not syncing in 8 - 12 hours but not in the official documentation. But, that’s fine.

The scenario is, that I added a calendar via URL. This URL is a RFC 5545 compliant iCalendar resource that returns a couple of events, including some recurring events. The resource is valid according to the validator on icalendar.org. And indeed, the very first import in Google calendar looked correct. At some point, I noticed that new events didn’t appear anymore. The reason was, that I got the folding wrong (for the description) and the actual resource was invalid and therefore Google calendar didn’t add the new events. It just ignored the whole ical resource. Once I fixed this the new events started to appear again after a while. So far so good.

But some time later, I noticed, that a specific change to an event didn’t show up. The change was a different recurrent rule, so it was obvious that something was not working: The old event was from Monday to Thursday. But it has been changed to repeat weekly now from Monday to Friday. And the Friday calendar entry was missing in the Google calendar. Which is pretty obvious in the weekly view.

Caching?

I checked the access logs and there is a regular entry from a User Agent called “Google-Calendar-Importer” that accesses the calendar. The interval was between 4 hours and 7 hours. So, really within the boundary of 8-12 hours mentioned above. But I double checked, whether I return any caching related HTTP headers - and I didn’t. So in theory, it could very well be the case, that the whole resource is somehow cached. Although then I should see any access at all. But just to be sure, I added the headers Last-Modified and Cache-Control: max-age=3600, must-revalidate. That means, it can be cached, but it needs to be revalidated on every request (e.g. via a HEAD request). If the last modification timestamp didn’t change, the cached value can be used. I also implemented the possibility to execute a HEAD request, which wasn’t possible before.

I experimented a bit: When I change the URL and add some query parameter, in theory I should by-pass all caching infrastructure. A GET request with a query-parameter shouldn’t be cached at all. And indeed, this worked: The imported events from that modified URL appeared correctly.

I also tried to remove the subscription and add it back again (with the same URL). But that didn’t show success: It appeared the same way as before - the one event with the changed recurrent rule was still the old version. All other events showed up as before. I have even seen a fresh request in the access log. That means, that Google calendar actually requested the calender, but didn’t use it? Because there is another cache going on?

Caching single events… change management

It took me a while to notice that Google calendar seems to cache at the level of a single event. Clearly the calendar is requested and downloaded. New events appear in the calendar, but previously existing events are not updated.

That led to back to the specification again for iCalendar. There is actually a section on 3.8.7 Change Management Properties. This explained a lot: For each event, a couple of properties can be defined, such as “CREATED” (when the event was created), “DTSTAMP” (a timestamp of creation or last revision), “LAST-MODIFIED” (when the event was last revised). And lastly “SEQUENCE” - which is kind of a incrementing version counter for every revision of the event.

I checked, which of these properties I change when the event is changed. And first I only changed the “LAST-MODIFIED” time stamp. I kept the “DTSTAMP” the same, since I interpreted the specification in that way, that this time stamp says when this event was first published in the calendar (the METHOD I used is publish…). I experimented with that to use the same value as LAST-MODIFIED (since it could be, that the Google devs interpreted this section of the specification in a different way), but that didn’t work. But I still now return the last modification time stamp for that.

The last property is “SEQUENCE” and I implemented it that way, that I always returned “0”. And I never returned anything else. So, as last step, I implemented a simple change counter that gets incremented whenever the event is modified in some way. And - lo and behold - it immediately started working. Immediately meaning the next time Google calendar synchronized, which was a couple of hours later. But then, the event suddenly appeared additionally on Friday, as it was intended.

Note, that Google Calendar seems to manage the events based on a combination of the URL of the imported calendar and the event’s UID (Unique Identifier). That means, you really need to make sure, that the UID of an event doesn’t change and is stable, but still unique. Otherwise you might see duplicated events or you override existing events during import.

I would also expect, that this caching/event management that Google Calendar does in the background is actually independent of the current user’s calendar: If two (or more) users import a calendar from the same URL, they would need to import this calendar only once. But that’s just an idea and I didn’t try to verify this. If that’s true, then - in theory - you shouldn’t see additional access requests on the URL if more Google users import that calendar into their own. As Google would then do only one request for all users.

Summary

This means, that maybe some of the complaints about Google calendar not syncing in time or never syncing like Google Calendar does not sync URL-linked calendars within 12 hours as stated or Why is my Google Calendar sync slow or not updating at all? or Intergrated iCal calendar not syncing in 8 - 12 hours might actually be a result of a not compliant iCalendar source, that doesn’t properly maintain the change management properties. And Google Calendar might actually just work correctly.

There is of course one challenge still open, that’s asked here: How do I adjust sync frequency in google calendar?. When you import a iCalender via URL, you have no control over how often Google Calender checks for updated / new events. If it is not so time critical, then the stated “8-12 hours” time frame might be ok. If not, there is another solution available as GAS-ICS-Sync. This is a script that runs on Google’s App Script. I can be configured to run at an arbitrary interval and it actively downloads the iCalendar resources and uses Google Calendar API to add/update calendar events.

For me, it was enough to fix my custom built iCalendar generator to correctly export the events with the correct property SEQUENCE.