New domain and migrating data between Google Apps

Short version

  • If you can’t reach me at my @cgoncalves.info, use @cgoncalves.pt from now on.
  • Google Apps doesn’t allow one to switch a primary domain with an alias. Had to migrate all data by myself…

A not so short version

Holding the cgoncalves.info domain since 2007, I had it associated to a free Google Apps account ever since.

Recently FCCN (Fundação para a Computação Científica Nacional) decided well it was time to liberalize the .PT TLD to everyone and not just to companies or trademarked brands. Now for ~17€/year any individual can get a .PT domain, no bureaucracy in the way. Hurray!

I did take the opportunity and acquired the cgoncalves.pt domain. It’s simple, elegant and somewhat represents pride of being a Portuguese citizen. This domain’s just what was pending to fulfill, as of now, the digital image of myself.

With this new domain I could add it to my existing Google Apps account and make it an alias of cgoncalves.info. Although as I’m so existed and convinced I’ll keep this new domain till my death, I gave a thought (2 seconds, if much) and concluded I should settle with the domain as the primary one instead of the rather deprecated cgoncalves.info. “So what?” you ask. Well… Google doesn’t allow their Google Apps users to switch a primary domain with an alias! Best chance to accomplish this is to create a new Google Apps account associated with the .PT domain and migrate all data from one account to the other, including emails, filters, contacts, calendars, feeds, etc. Moving contacts, calendars and feeds is a trivial action but hundred of thousands of emails from years?! No fun…

Once migration completed, delete the cgoncalves.info Google Apps account and add that domain to the new account as alias was the goal to achieve.

Migrating emails

I started migrating emails between the two Google Apps accounts using the POP3 fetcher GMail provides. Yes, it take ages and only fetches up to 200 emails at a time, but that was the faster way I know. The fetcher automatically retrieves for messages at different rates from accounts and depends on previous mail fetch attempts (the less mail retrieved, the less frequency it will check for new mail). Frustratedly even with thousands of messages yet to get, the intervals were set longer and longer (sometimes to two hours!)… Enabling “Refresh POP accounts” lab (“Fetch messages from your POP accounts on demand by using the refresh link on to of the inbox”) would cut the time between fetches but was no usable solution as it required human interaction by clicking on the gear icon now and then.

I also tried this script and threw one dollar (minimum donation amount required) to let me unlock the fixed 8 minutes interval and empower me to set it to a higher rate (down to 3 minutes). It was working like a charm and the migration process was flying fairly decently. BUT NO! OH NO! GMail stopped successfully retrieving messages and started emailing me an unnumbered “Message left on server” messages. Perhaps the daily quota has been reached, or not as it kept that way the day after. This POP3-me thing was running for now a full week and I still had many messages left to migrate. I was doomed!

With the POP3 process discarded there was just one option left and that I was battling to avoid: transfer from IMAP. Don’t get me wrong, I love and use IMAP in all my accounts. The side of the coin is I’d have to perform the migration on my end and reached the first 500 messages transferred, only 1 message per second is synced. Imapsync was just the application suited to do the job. It detects duplicated emails and all, great! I installed it on my VPS (so I can put my laptop to sleep at will) and ran it as follows:

#!/usr/bin/env bash
imapsync \
--host1 imap.gmail.com \
--user1 account@old-domain.com \
--passfile1 ./passfile1 \
--ssl1 \
--host2 imap.gmail.com \
--user2 account@new-domain.com \
--passfile2 ./passfile2 \
--ssl2 \
--authmech1 LOGIN \
--authmech2 LOGIN \
--useheader 'Message-Id' \
--useheader 'X-Gmail-Received'

Files passfile1 and _passfile2 contain the respective email accounts passwords (plain text). And voilá! It might take several hours/days but I didn’t know (and still don’t) no better solution. And remember, POP3 mail fetcher doesn’t copy sent mail, but _imapsync does! Win!

Back to Google Apps drama

At last with data migrated I let things stabilize for a couple of days before taking any further action. Next step was ensuring I did not depend anymore on my @cgoncalves.info Google Apps account: check! Logged into the Google Apps administration panel of the new domain and tried to add cgoncalves.info as an alias. It failed with something like “The domain is currently associated to another account”. That was right, I still had to purge the former account before proceeding: done! Retried once again hoping I got lucky but no… “This domain name has already been used as an alias or domain. It seems I will have to wait up to seven days before I can add the .INFO domain as an alias.

Conclusion

Update my contact record on your address book and replace everything ending with cgoncalves.info to cgoncalves.pt. I promise I’ll keep using this domain for decades, unless someone steps in with a bag full of money and buys me off to concede the ownership of the domain. Starting bid is 1000€. I mean it! :-D