Monday, January 25, 2010

Steps to creating a EBS image from an AMI (Windows)

Download Amazon's EC2 API Tools to C:\ec2-api-tools

# Run the following on a remote machine with Amazon's EC2 API Tools installed
# Adjust these variables to match your own system
set EC2_HOME=c:\ec2-api-tools
set JAVA_HOME=c:\Program Files\Java\jre6

# Update PATH
set PATH=%PATH%:c:\ec2-api-tools\bin

#Test by running
ec2-version
# Output: 1.3-46266 2009-11-30

# Add keys
set EC2_PRIVATE_KEY=c:\aws-keys\pk-XXXXXXX.pem
set EC2_CERT=c:\aws-keys\cert-XXXXXXX.pem

# Use AWS Management Console
# Create new instance based on existing AMI
# Once instance is launch and running create a volume, make sure that the availability zone is the same as the instance, set device to /dev/sdh
# Attach the volume to the instance
# Connect to the instance using Putty
# Create the following script
vi instance-to-ebs-ami.sh

#!/bin/bash
# Run this script on the instance to be bundled
# tested with Canonical Ubuntu 9.10 base ami

EBS_DEVICE=${1:-'/dev/sdh'}
IMAGE_DIR=${2:-'/mnt/tmp'}
EBS_MOUNT_POINT=${3:-'/mnt/ebs'}

mkdir -p $EBS_MOUNT_POINT
mkfs.ext3 ${EBS_DEVICE}
mount ${EBS_DEVICE} $EBS_MOUNT_POINT

#make a local working copy
mkdir /mnt/tmp
rsync --stats -av --exclude /root/.bash_history --exclude /home/*/.bash_history --exclude /etc/ssh/ssh_host_* --exclude /etc/ssh/moduli --exclude /etc/udev/rules.d/*persistent-net.rules --exclude /var/lib/ec2/* --exclude=/mnt/* --exclude=/proc/* --exclude=/tmp/* / $IMAGE_DIR

#ensure that ami init scripts will be run
chmod u+x $IMAGE_DIR/etc/init.d/ec2-init-user-data

#clear out log files
cd $IMAGE_DIR/var/log
for i in `ls ./**/*`; do
echo $i && echo -n> $i
done

cd $IMAGE_DIR
tar -cSf - -C ./ . | tar xvf - -C $EBS_MOUNT_POINT
#NOTE, You could rsync / directly to EBS_MOUNT_POINT, but this tar trickery saves some space in the snapshot

umount $EBS_MOUNT_POINT

chmod +x instance-to-ebs-ami.sh
./instance-to-ebs-ami.sh
exit

# Go back to AWS Management Console
# Detach the volume
# Create a snapshot of the volume

# Now, we register the snapshot to be able to create an EBS image
# Run this command from your remote machine with Amazon's EC2 API Tools installed
# Fill in the required variables

ec2-register --snapshot snap-xxxxxx --kernel aki-xxxxxxx --ramdisk ari-xxxxxxxx --description="" --name="debian-5.0-lenny-base-20091011" --architecture i386 --root-device-name /dev/sda1
#sample output => IMAGE ami-xxxxxxx


References/Credits:
http://www.capsunlock.net/2009/12/create-ebs-boot-ami.html
http://gist.github.com/249915

Try this script for creating automated snapshots:
http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=100&externalID=1663
or
http://johnwoconnor.blogspot.com/2009/06/automatic-volume-snapshots-using-amazon.html

Friday, July 17, 2009

Creating a Remote Git Repo

Here are some steps I used to create a remote Git repo based on my local files.

Create the folder for the new application and change to that directory.
> mkdir /var/www/html/newapp
> cd /var/www/html/new
app

Make sure git is installed, if not install it
> sudo apt-get install git-core git-doc

Initialize the folder to become a git directory
> git init

********* Now switch to your local files ***********
> cd ~/myapp

Initialize git with the local folder as well
>git init

Tell git about yourself
>git config user.name "Your Name"
>git config user.email "Your Email"


Now, tell git which server to talk too
>git remote add origin username@yourdomain.com:/var/www/html/newapp/.git
Make sure that this user has SSH access to the box.


>git config branch.master.remote origin
>git config branch.master.merge refs/heads/master


Make your initial commit and push it to the remote server.
>git commit -a -m "Initial commit"
>git push origin master


And viola! Your files will be pushed to the remote server. Now, I was wondering why when I went to access the server, the files didn't seem to be in the working directory, even though I could see the log showed my Initial commit. It turns out that when things are pushed to master, they enter a limbo state. You have to do a git checkout -f, to actual have the commit become live. Note that doing a git checkout -f, will throw away any changes that have been made on the remote server.

To make the updating of the server easier, you can do the following:
> vi /var/www/html/newapp/.git/hooks/post-receive
Add - git checkout -f
Save the file
>chmod +x /var/www/html/newapp/.git/hooks/post-receive

Friday, June 26, 2009

Read-optimize your source code

A good article about writing well-written code that will make it easy for other developers to quickly understand your code.

http://www.brendel.com/consulting/blog/2009/06/read-optimized-source-code.html

Another useful unix command to find text in many files

find . -type f -exec grep -il "text to find" {} \;

Will give you a list of all the files that contain "text to find", starting with your current directory and searching all sub-directories.

Tuesday, June 9, 2009

Useful unix command to find and replace text in many files

Here is a useful unix command I used today to find and replace text across multiple files and multiple directories.

find . -type f -exec sed -i "s/string1/string2/g" {} \;

This will replace all instances of string1 with string2.

Credit to Google and http://www.sb.fsu.edu/~soma/Proj/How-to/Search+Replace.pdf for helping me figure this out.

Wednesday, May 6, 2009

Koha April, 7th meeting

Galen plans to do a feature freeze at the end of August and release 3.2 at the end of October.

Before June we should be able to test BibLibre's acq work and Liblime will be reporting on there development work.

Galen will be doing a Release Manager blog for Koha 3.2. If Nicole has time she will take a first stab at improving the Koha User list welcome message to include a good Koha FAQ and links to helpful tips and guides.

hdl mentioned that Bugzilla can accept emails...interesting, I will have to look into this and see if it can improve my workflow in tracking and dealing with bugs.

All new enhancements will be tracked using Bugzilla in order to keep everything in one place. This will help the community be better informed about what other Koha libraries are sponsoring and what developers are working on.