tag:blogger.com,1999:blog-49682932214856858512024-02-08T02:51:48.696-08:00cnpbagwellcbagwellhttp://www.blogger.com/profile/17622517055195672964noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-4968293221485685851.post-40138710026537405982009-09-26T16:57:00.000-07:002009-09-26T16:57:57.011-07:00Migrating from Gmail to Google AppsNotes taken while I migrated from Gmail+Calendar+Contacts to Google Apps with equivalent features.<br />
<br />
<span style="font-size: large;">Migrate Emails</span><br />
<br />
I used imapsync on Fedora 11 to do the dirty work of transferring emails from Gmail to Google Apps. I used the script as defined by <a href="http://www.thamtech.com/blog/2008/03/29/gmail-to-google-apps-email-migration/">this blog</a>. I made some customizations to ease switching between multiple users that I transferred to Google Apps.<br />
<br />
If you have access to a semi-recent version of Fedora you can install imapsync with "yum install imapsync".<br />
<br />
Save the following script text to a file and "chmod 755 filename.sh". Update the USER1 and USER2 lines with your account names. You'll need to create two files called passfile1 and passfile2 that contain the passwords for each account.<br />
<br />
Note: Compared to script from above log, I added --allowsizemismatch to get past some emails that somehow had wrong file sizes stored. You may wish to try without that option first.<br />
<br />
--- begin script ----<br />
<span style="font-family: "Courier New",Courier,monospace;">#!/bin/bash</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">USER1='user@gmail.com'</span><br />
<span style="font-family: "Courier New",Courier,monospace;">USER2='user@domain.com'</span><br />
<span style="font-family: "Courier New",Courier,monospace;">USER1E=`echo ${USER1} | sed 's/@/\\\\@/'`</span><br />
<span style="font-family: "Courier New",Courier,monospace;">USER2E=`echo ${USER2} | sed 's/@/\\\\@/'`</span><br />
<span style="font-family: "Courier New",Courier,monospace;">PASSFILE1=./passfile</span>1<br />
<span style="font-family: "Courier New",Courier,monospace;">PASSFILE2=./passfile2</span><br />
<span style="font-family: "Courier New",Courier,monospace;">LOGFILE=imapsync.log</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">echo "***** Starting *****" > $LOGFILE</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">echo "" >> $LOGFILE</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">time imapsync --host1 imap.gmail.com \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --port1 993 --user1 ${USER1} \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --passfile1 $PASSFILE1 --ssl1 \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --host2 imap.gmail.com \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --port2 993 --user2 ${USER2} \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --passfile2 $PASSFILE2 --ssl2 \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --syncinternaldates --split1 100 --split2 100 \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --authmech1 LOGIN --authmech2 LOGIN \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --useheader "Message-ID" \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --useheader "Date" --skipsize --allowsizemismatch \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --regexmess "s/Delivered-To: ${USER1E}/Delivered-To: ${USER2E}/gi" \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --regexmess 's/Subject:(\s*)\n/Subject: (no--subject)$1\n/ig' \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> --regexmess 's/Subject: ([Rr][Ee]):(\s*)\n/Subject: $1: (no--subject)$2\n/gi' 2>&1 | tee -a $LOGFILE</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">echo "" >> $LOGFILE</span><br />
<span style="font-family: "Courier New",Courier,monospace;">echo "***** COMPLETE *****" >> $LOGFILE</span><br />
<span style="font-family: "Courier New",Courier,monospace;">echo "" >> $LOGFILE</span><br />
--- end script ---<br />
<br />
<span style="font-size: large;">Migrate Calendars</span><br />
<br />
Transferring your Primary calendar is hardest part. If you have multiple calendars, the non-primary calendars can simply be Shared with the new Google Apps account and you will see no differences. For your primary calendar, I found it easiest to save an ical version of events and then import it.<br />
<br />
Use these steps to save save an ical version of your events from Gmail: Under "My Calendars" list, click the Settings. Under Settings, click on your primary calendar. Towards bottom of new screen lists you "Private Address". On the green ICAL icon, right click and click your Save Link As option (using Firefox anyways).<br />
<br />
On your Google Apps calendar, click the Settings on your "My Calendars" list. On next screen, there is an option to Import Calendar. Select it and when prompted, give the filename of your ICAL saved in previous step.<br />
<br />
<span style="font-size: large;">Migrate Contacts</span><br />
<br />
The follow info on contacts was taken from a <a href="http://www.base6.com/2009/06/15/migrating-from-gmail-to-google-apps/">blog also on migrating</a>.<br />
<br />
To preserve contact groups, each group needs to be migrated individually.<br />
<ol><li>On Google Apps mail, pre-create all your Groups that existed under Gmail. <br />
</li>
<li>On GMail contacts, use the Export option to export each group in Google’s CSV format. Then export one more CSV of “Everyone (All Contacts)”</li>
<li>On Google Apps contacts: <br />
<br />
<ol><li>Use the Import option to import the CSV of Everyone, leaving the “Add these imported contacts to: ” option unchecked.</li>
<li>Then use the Import option to import each group in turn, checking the “Add these imported contacts to:" and making sure to select the correct group name that was created in step #1. </li>
</ol><br />
<br />
</li>
</ol><span style="font-size: large;"><span style="font-size: small;"><span style="font-size: large;">Migrate iPhone</span></span></span><br />
<br />
<span style="font-size: small;">If you were using Google's Sync option for iPhone using Active Sync you were already pointing at the correct servers. Just need to change Username from user@gmail.com to new user@domain.com and also the Password.</span><br />
<span style="font-size: small;"> <br />
</span><br />
<span style="font-size: small;">To be safe, I toggled OFF and then back ON the settings for Mail, Contacts, and Calendar and also selected "Delete from iPhone" when prompted. This did a force resync of these items although calendar ended up empty.<br />
</span><br />
<span style="font-size: small;"><br />
</span><br />
<span style="font-size: small;">You will also need to use Safari and browse to m.google.com/sync. Look for instructions for configuring "Google Apps" and follow them. You will need to do this to switch google sync to use your Google Apps calendars.<br />
</span>cbagwellhttp://www.blogger.com/profile/17622517055195672964noreply@blogger.com0tag:blogger.com,1999:blog-4968293221485685851.post-4907681649329749242009-03-08T13:09:00.000-07:002009-09-13T13:13:11.826-07:00Backing up Windows Vista with Free Software<div style="font-family: inherit;">I’ve been attempting to deal with the sad state of backup affairs that Windows Vista is in. I have one computer with a spare hard drive that I could easily dump files to. This computer is running Vista Basic which contains a backup program that can only be manually invoked and which only does partial backups. I have another computer running Vista Home Premium which can automate backups but has no spare hard drive and I’m suffering from well documented issues with Vista and sharing remote disks.</div><div style="font-family: inherit;">I looked for free or cheap programs to help me automate this task but most would have been difficult to recover from; as they used ZIP archives. I preferred a “rsync” based solution as thats what I use under Linux with great success. So here are the steps I used to get it working. This is a collection of bits of information I found all over the web.</div><h2><span style="font-size: small;">Cygwin Setup</span></h2>The first step was to install <a href="http://www.cygwin.com/">Cygwin</a>. Using the Cygwin SETUP.EXE, make sure and install the “rsnapshot” program. This program automates using rsync to backup the hard drive (using hard links to save space) and also doing the housework of rotating directories over time.<br />
It is *very* important that after installing cygwin that you tell it to not use NT-style security. You will run into unsolvable file permission issues with your backed up files unless you do this next step. Navagate to My Computer->Properties->Advanced->Environment Variables->System Variables. Either edit or add “<code>CYGWIN = nontsec tty".</code><br />
Another option would be to investigate <a href="http://www.itefix.no/i2/node/10650">cwrsync</a>. It basically automates the above steps for you; installing a mini-cygwin environment within its own directory tree.<br />
<h2><span style="font-size: small;">Shadow Copy Support</span></h2>All versions of Vista include support for creating Shadow versions of drives. This allows you to see a file system in a consistent state at some exaxt point in time. I believe it also allows you to open/read files that applications already have opened. Window’s Backup and most commercial backup programs make us of this feature.<br />
Microsoft has a command line program that allows you to hook into this feature. You can request to create a shadow copy of a drive and mount it to a drive letter. I’ve found that backing up a shadow drive is infinitely more stable then backing up an live drive.<br />
The steps later require installing this command line program. Install the <a href="http://www.microsoft.com/downloads/details.aspx?FamilyID=0B4F56E4-0CCC-4626-826A-ED2C4C95C871&displaylang=en">Volume Shadow Copy Service SDK</a>.<br />
<h2><span style="font-size: small;">rsnapshot setup</span></h2>Edit /etc/rsnapshot.conf to your liking. Here are some options that I used to backup my C: drive (shadowed as a J: drive) to my F: drive .<br />
<blockquote><pre>snapshot_root /cygdrive/f
exclude_file /etc/rsnapshot.exclude
logfile /cygdrive/f/snapshot.log
one_fs
no_create_root 1
backup /cygdrive/j/ localhost/
</pre></blockquote>Next, you’ll need to create the file /etc/rsnapshot.exclude file that is referenced above. It contains a list of Vista’s default junction points and will prevent copying the same file multiple times and prevent infinit recursion error messages from being printed.<br />
<blockquote><pre>#References:
#http://codydunne.blogspot.com/2008/09/rsyncd-exclusions-for-using-backuppc-on.h
tml
#http://www.svrops.com/svrops/articles/jpoints.htm
#Junction points or symlinks
- /cygdrive/j/Users/All Users
- /cygdrive/j/Users/Default User
- /cygdrive/j/Users/All Users/Application Data
- /cygdrive/j/Users/All Users/Desktop
- /cygdrive/j/Users/All Users/Documents
- /cygdrive/j/Users/All Users/Favorites
- /cygdrive/j/Users/All Users/Start Menu
- /cygdrive/j/Users/All Users/Templates
- /cygdrive/j/Users/Public/Documents/My Music
- /cygdrive/j/Users/Public/Documents/My Pictures
- /cygdrive/j/Users/Public/Documents/My Videos
- /cygdrive/j/ProgramData/Application Data
- /cygdrive/j/ProgramData/Desktop
- /cygdrive/j/ProgramData/Documents
- /cygdrive/j/ProgramData/Favorites
- /cygdrive/j/ProgramData/Start Menu
- /cygdrive/j/ProgramData/Templates
- /cygdrive/j/Documents and Settings
#Excludes these junction points common to every user profile
- /cygdrive/j/Users/*/Application Data
- /cygdrive/j/Users/*/Cookies
- /cygdrive/j/Users/*/Local Settings
- /cygdrive/j/Users/*/My Documents
- /cygdrive/j/Users/*/NetHood
- /cygdrive/j/Users/*/PrintHood
- /cygdrive/j/Users/*/Recent
- /cygdrive/j/Users/*/SendTo
- /cygdrive/j/Users/*/Start Menu
- /cygdrive/j/Users/*/Templates
- /cygdrive/j/Users/*/AppData/Local/Application Data
- /cygdrive/j/Users/*/AppData/Local/History
- /cygdrive/j/Users/*/AppData/Local/Temporary Internet Files
- /cygdrive/j/Users/*/Documents/My Music
- /cygdrive/j/Users/*/Documents/My Pictures
- /cygdrive/j/Users/*/Documents/My Videos
#Temporary and in-user user data
- /cygdrive/j/Users/*/AppData/Local/Microsoft/Windows/Temporary Internet Files
- /cygdrive/j/Users/*/AppData/Local/Temp
- /cygdrive/j/Users/*/NTUSER.DAT
- /cygdrive/j/Users/*/ntuser.dat.LOG1
- /cygdrive/j/Users/*/ntuser.dat.LOG2
- /cygdrive/j/Users/*/AppData/Local/Microsoft/Windows/UsrClass.dat
- /cygdrive/j/Users/*/AppData/Local/Microsoft/Windows/UsrClass.dat.LOG1
- /cygdrive/j/Users/*/AppData/Local/Microsoft/Windows/UsrClass.dat.LOG2
- /cygdrive/j/Users/*/AppData/Local/Microsoft/Windows Defender/FileTracker
- /cygdrive/j/Users/*/AppData/Local/Mozilla/Firefox/Profiles/*/Cache
- /cygdrive/j/Users/*/AppData/Roaming/Microsoft/Windows/Recent
- *.lock
#Installation folders and system data
/cygdrive/j/Program Files
/cygdrive/j/Windows
/cygdrive/j/$Recycle.Bin
/cygdrive/j/MSOCache
/cygdrive/j/System Volume Information
/cygdrive/j/autoexec.bat
/cygdrive/j/bootmgr
/cygdrive/j/BOOTSECT.BAK
/cygdrive/j/config.sys
/cygdrive/j/hiberfil.sys
/cygdrive/j/pagefile.sys</pre></blockquote><h2><span style="font-size: small;">Scripts to perform copy</span></h2>The following script should be used as a template. You will need a script for each “interval” line in your /etc/rsnapshot.conf and it needs to pass in “hourly”, “daily”, “weekly”, or “monthly” to the rsnapshot program.<br />
The script will create a shadow version of the C: drive and map it to drive J:. It will then invoke rsnapshot to backup the J: drive and afterwards delete the showdow drive.<br />
<blockquote><pre>#!/bin/sh</pre><pre>### Change to temporary directory so that we have a known location</pre><pre>### of temporary files created in local directory.</pre><pre>cd /tmp</pre><pre>### Create shadow drive of C: and store off its ID</pre><pre>###</pre><pre>/cygdrive/c/Program\ Files/Microsoft/VSSSDK72/TestApps/vshadow/bin/release-serve</pre><pre>r/vshadow.exe -p -nw -script=vars.bat c:</pre><pre>SHADOW_ID_1=`grep SHADOW_ID_1 vars.bat | sed "s/.*=//"`</pre><pre>### Map to drive J:</pre><pre>###</pre><pre>/cygdrive/c/Program\ Files/Microsoft/VSSSDK72/TestApps/vshadow/bin/release-serve</pre><pre>r/vshadow.exe -el=$SHADOW_ID_1,J:</pre><pre>### Run backup</pre><pre>###</pre><pre>/usr/bin/rsnapshot daily >> /cygdrive/f/rsnapshot.stdout 2>&1</pre><pre>### Delete shadown drive</pre><pre>###</pre><pre>/cygdrive/c/Program\ Files/Microsoft/VSSSDK72/TestApps/vshadow/bin/release-serve</pre><pre>r/vshadow.exe -ds=$SHADOW_ID_1</pre><pre>### Go back to temporary directory and clean up files.</pre><pre>###</pre><pre>cd /tmp</pre><pre>rm vars.bat</pre></blockquote><h2><span style="font-size: small;">Scheduling Backups</span></h2>You can use Window’s Scheduled Tasks tool to run the above script at the intervals of your chosing. Start->Accessories->System Tools->Scheduled Tasks.<br />
The command line to use is basic form shown below. Replace with the filename you created using above script template.<br />
<blockquote><pre>c:\cygwin\bin\bash.exe /path/to/rsnapshot_xxx.sh</pre></blockquote>Note: Just because you run the “weekly” interval on a given day each week, you still need to run the “daily” interval that same day. Same thing applies for all intervals.cbagwellhttp://www.blogger.com/profile/17622517055195672964noreply@blogger.com0tag:blogger.com,1999:blog-4968293221485685851.post-84781562374219611822009-03-08T13:05:00.000-07:002009-09-13T13:07:14.471-07:00Automating emails to cancel iCalendar eventsIn a follow up to my post about emailing <a href="http://www.cnpbagwell.com/blog/?p=3">iCalendar requests</a>,you may wish to send out an automated email to cancel that request and have the user automatically be prompted to remove it as well. That can be handled with minor modification to the previous script and .ics files.<br />
<br />
First, find the following piece of code:<br />
<blockquote><pre>icspart = MIMEBase('text', 'calendar', **{'method' : 'REQUEST', 'name' : 'meeting.ics'})</pre></blockquote>and change that to a CANCEL method:<br />
<blockquote><pre>icspart = MIMEBase('text', 'calendar', **{'method' : 'CANCEL', 'name' : 'meeting.ics'})</pre></blockquote>Next, in the original .ics file, change any line with <strong>METHOD:</strong> to read <strong>METHOD:CANCEL</strong> and any line with <strong>STATUS:</strong> to read <strong>STATUS:CANCELLED</strong>.cbagwellhttp://www.blogger.com/profile/17622517055195672964noreply@blogger.com0tag:blogger.com,1999:blog-4968293221485685851.post-83576586813500791712009-01-22T12:50:00.000-08:002009-09-13T13:05:23.458-07:00Automating emails of iCalendar (.ics) filesI was surprised after a period of googling that I couldn’t find a python script to email iCalendar files. More specifically, I was wanting to email a *.ics file as an attachment and if it was received by an Outlook client I would like for Outlook to automatically add the buttons to Accept/Deny/etc just like when sent from Outlook itself.<br />
<a href="http://www.mavetju.org/programming/outlook-ics.php">I found a site</a> that did most the research on this task already but they used Perl. Also, their script creates the email but doesn’t send it.<br />
I wanted to use python since it had all included modules already to both create the message and send it using SMTP. For your pleasure, here is my version of the script.<br />
<pre><blockquote>#!/usr/bin/python
import smtplib
import os
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email.Utils import COMMASPACE, formatdate
from email import Encoders
send_from = "your_email@yourdomain.com"
send_to = "your_email@yourdomain.com"
subject = 'Meeting Invite'
msg = MIMEMultipart('alternative')
msg['From'] = send_from
msg['To'] = send_to
msg['Date'] = formatdate(localtime=True)
msg['Subject'] = subject
msg.add_header('Content-class', 'urn:content-classes:calendarmessage')
msg.attach(MIMEText("See attachement for Meeting Invite."))
icspart = MIMEBase('text', 'calendar', **{'method' : 'REQUEST', 'name' : 'meeting.ics'})
icspart.set_payload( open("meeting.ics","rb").read() )
icspart.add_header('Content-Transfer-Encoding', '8bit')
icspart.add_header('Content-class', 'urn:content-classes:calendarmessage')
msg.attach(icspart)
#print msg.as_string()
smtp = smtplib.SMTP('localhost');
smtp.sendmail(send_from, send_to, msg.as_string())
smtp.close()</blockquote></pre>This requires an iCalendar file called meeting.ics to already exist in current directory.<br />
This script does the minimum work possible and leaves plenty of room for improvements. For example, it would be nice to have text body of the email be the same as DESCRIPTION in the meeting.ics file so that user can see what the calendar item is about even if their application doesn’t support *.ics files. And it could base Subject off of SUMMARY.<br />
From other web site, it sounds like the email sent isn’t compatible with Mac OSX iCal because they want PUBLISH instead of REQUEST. I guess we can’t please everyone right now.cbagwellhttp://www.blogger.com/profile/17622517055195672964noreply@blogger.com0