Eucalyptus Installation

!! Overview These are the steps that were taken to create a functional Eucalyptus 3.1 system. Theses intstructions are for CentOS 6.1. They may or may not work on other distributions. (Definitely won't work for any non-RedHat OS)

!! Hardware This guide assumes that you have 2 or more computers to use in order to build a Eucalyptus cloud. These computers must support hardware virtualization (Read the "Prepare System" section for more info). This guide will be documenting how to set up a Eucalyptus cloud using the MANAGED networking mode. In our case, this required the machine that is running the Cluster Controller (CC) to have a second Network Interface Card (NIC) so that all Node Controllers (NCs) were on their own subnet.

!! Prepare System We need to do a few things to make sure that our system is ready for Eucalyptus to be installed.

Update the BIOS is one of the things we needed to do, Here is how you can do it too:

Step 1: Downloading FreeDOS to use to boot the machine and run the BIOS .EXE

wget http://www.fdos.org/bootdisks/autogen/FDOEM.144.gz Then, gunzip FDOEM.144.gz to unzip the file.

Step 2: Copy the BIOS flash utility and the BIOS image that needs to be used to upgrade, and mount it to a floppy disk image. Here is what you do in order to complete this step:

modprobe vfat

modprobe loop

mkdir /tmp/floppy

mount -t vfat -o loop FDOEM.144 /tmp/floppy

After mounting the Floppy you want to copy the EXE. that you downloaded for the BIOS:

cp DELLBIOSVERSION.exe /tmp/floppy (not actual name of the bios)

then unmount the floppy: umount /tmp/floppy

Step 3 is to burn the bootable CD which emulates a floppy device.

mkisofs -o bootcd.iso -b FDOEM.144 FDOEM.144 cdrecord -v bootcd.iso

Now you should have a bootable CD that can boot into FreeDOS where you can then run the BIOS .EXE that you have put on it.

!!! Disable SELinux Eucalyptus does not work with SELinux (Security Enhanced Linux). Disable it! Open the following file in your favorite text editor (I use vim): vim /etc/selinux/config

Change the following line: SELINUX=enforcing To this: SELINUX=permissive

Then run: setenforce 0

!!! Open Ports (or Disable Firewall) You have a choice here. You can either open the following ports by editing iptables, or just turn off the firewall. If you want to add firewall rules to allow traffic on those ports, the easiest way is by running... system-config-firewall-tui ... and following the prompts to open the following ports on the machines you will be using for the CLC/CC:
 * 1) 8443
 * 2) 8773
 * 3) 8774
 * 4) 9001

Open port 8775 on all Node Controllers.

Don't feel like opening ports? Select the option to disable the firewall in `system-config-firewall-tui` and move on with your life.

!!! Check for Hardware Virtualization We are going to be using KVM (Kernel-based Virtual Machine) for our virtual machine instances, as is this is what RedHat is supporting now rather than Xen. KVM requires that your CPU have hardware virtualization support (Intel VT or AMD-V) in order to work.

To check for that, run the following command: egrep '^flags.*(vmx|svm)' /proc/cpuinfo

If there is NO output then your CPU does NOT have hardware virtualization support, and therefore cannot use KVM. If you're willing to sell your soul you may still be able to use VMWare...

!!! Sync with NTP Time Server The system clocks need to synchronized across all machines. Install the Network Time Protocol daemon for this. Install the ntp package if it is not already installed: yum install -y ntp

Sync with time server: ntpdate pool.ntp.org

Start the service: service ntpd start

!! Install Dependencies

!!! Install Additional Repositories

Download the following .rpm package files More info [| here].

wget http://yum.pgrpms.org/9.1/redhat/rhel-6-i386/pgdg-centos91-9.1-4.noarch.rpm wget http://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpm wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm wget http://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-1.el.noarch.rpm yum localinstall pgdg-centos91-9.1-4.noarch.rpm epel-release-6-5.noarch.rpm &#092; elrepo-release-6-4.el6.elrepo.noarch.rpm &#092; eucalyptus-nightly-release-3-1.el.noarch.rpm

!!! Necessary Manual Edit Note: DO THIS IF AND ONLY IF YOU ARE RUNNING A 32 BIT VERSION OF CENTOS.

It is HIGHLY recommended that you use a 64 bit OS, but making this change allows you to get some .noarch packages from the Eucalyptus repository. Otherwise, the package list will be 404 Not Found. Edit /etc/yum.repos.d/eucalyptus-nightly-release.repo This is the file for the Eucalyptus 3 development repositories. The x86_64 repos had to be used because the i686/i386 repositories gave a 404 error. [euca-3-devel] name=euca-3-devel baseurl=http://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/$releasever/x86_64/ gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLY gpgcheck=1 enabled=0

[euca-3-deps] name=euca-3-deps baseurl=http://downloads.eucalyptus.com/devel/packages/3-devel/centos/$releasever/x86_64/ gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLY gpgcheck=1 enabled=1 The changes that should be made are in bold.

!!! Install Packages Source: [] yum install bzr python-boto euca2ools libvirt-devel openssl-devel gcc ant ant-nodeps &#092; java-1.6.0-openjdk-devel curl-devel libxslt-devel xalan-j2-xsltc wsdl4j &#092; backport-util-concurrent httpd postgresql91-server libvirt PyGreSQL make &#092; openssh-clients scsi-target-utils qemu-kvm axis2-codegen axis2-adb-codegen &#092; httpd httpd-devel rampartc rampartc-devel axis2c axis2c-devel axis2

!!! Create eucalyptus User We need to add a eucalyptus user for the Eucalyptus services to run as. It is added to the kvm group so machines that are node controllers can manipulate instances. useradd eucalyptus passwd eucalyptus

!!! Modify WSDL2C.sh Open /usr/lib64/axis2c/bin/tools/wsdl2c/WSDL2C.sh in a text editor. Erase the existing lines and add the following:

#!/bin/sh

java -classpath $(build-classpath axis2/codegen axis2/kernel axis2/adb &#092;    axis2/adb-codegen wsdl4j commons-logging xalan-j2 xsltc &#092;     backport-util-concurrent ws-commons-XmlSchema ws-commons-neethi &#092;     ws-commons-axiom annogen ) org.apache.axis2.wsdl.WSDL2C $*

!! Compile Dependencies DON'T ACTUALLY DO THIS ON A 64 BIT MACHINE. The eucalyptus repository that was installed in a previous step has all of the packages you need. This is here for historical purposes and will probably be deleted soon. ''Note: Compilation only needs to happen on one machine. Once compiled, the binaries can be copied to the other servers''

This section is copied pretty much verbatim from the [| Eucalyptus 2.0 Installation Instructions].

!!! Download Source Code Download and extract tarball (2.0.3 dependencies work for 3.0 as far as I can tell) wget http://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-deps.tar.gz   tar -zxf eucalyptus-2.0.3-src-deps.tar.gz

Set shell variable for install location Dave's completely objective, non-biased Note: Installing to /opt is really ugly, but this what the Eucalyptus install guide uses, so I will too. export EUCALYPTUS=/opt/eucalyptus

Create directory for dependency binaries mkdir -p $EUCALYPTUS/packages/

Change directory to where the dependency sources were extracted Your path will probably be different cd ~/Downloads/eucalyptus-src-deps/

!!! Install Axis2 Extract and copy Axis2 No compilation here. Nice and easy. tar -zxf axis2-1.4.tgz mv axis2-1.4 $EUCALYPTUS/packages/

!!! Install Axis2C export APACHE_INCLUDES=/usr/include/httpd/ export APR_INCLUDES=/usr/include/apr-1/ export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.6.0

tar zxf axis2c-src-1.6.0.tar.gz   cd axis2c-src-1.6.0 CFLAGS="-w" ./configure --prefix=${AXIS2C_HOME} --with-apache2=$APACHE_INCLUDES --with-apr=$APR_INCLUDES --enable-multi-thread=no make make install

!!! Install Rampart/C export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib:$LD_LIBRARY_PATH

tar zxf rampartc-src-1.3.0-0euca2.tar.gz   cd rampartc-src-1.3.0 ./configure --prefix=${AXIS2C_HOME} --enable-static=no --with-axis2=${AXIS2C_HOME}/include/axis2-1.6.0 make make install

!!! Edit Rampart/C Configuration Change the following in $AXIS2C_HOME/axis2.xml. In the 'inflow' section, change: to 

In the 'outflow' section, change: to 

!! Compiling Eucalyptus

!!! Download Source Code Fetch the latest source from launchpad bzr branch lp:eucalyptus

Enter the source directory cd eucalyptus

!!! Compile Set shell variables for Java environment: export JAVA_HOME="/usr/lib/jvm/java-openjdk/" export JAVA="$JAVA_HOME/jre/bin/java"

Compile Eucalyptus ./configure --with-axis2c=/usr/lib64/axis2c/ --with-apache2-module-dir=/usr/lib64/httpd/modules/ make make install

!! Install Eucalyptus on Other Machines Based heavily upon this wonderful blog post.

!!! Copy Installation to All Machines Repeat for each machine. rsync -a $EUCALYPTUS/ root@:$EUCALYPTUS/ ( being the IP of the node you are copying to)

!!! Set environment variables These are needed pretty much everywhere. A good idea would be to place these in the ~/.bashrc files for root and eucalyptus users. You WILL have commands fail if you don't set these. export EUCALYPTUS=/opt/eucalyptus export PATH=$PATH:$EUCALYPTUS/usr/sbin

!!! Install Packages Follow the "Installing Additional Packages" and "Installing Dependencies" instructions at the top of this page to install needed packages onto the node

!!! Configure Edit /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf Find the following configuration variables and set the values accordingly EUCALYPTUS="/opt/eucalyptus" HYPERVISOR="kvm" USE_VIRTIO_DISK="1" USE_VIRTIO_NET="1" INSTANCE_PATH="/opt/eucalyptus/instances" VNET_BRIDGE="virbr0"

!!! Register Startup Scripts Create symbolic links to the daemons so that the `service` command will work with them ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc

!!! Edit Hosts A common problem that occurs is that the `euca_conf --initialize` command will fail. This usually happens because the system hostname isn't in the /etc/hosts file.

Edit hosts file: vim /etc/hosts

Add something like the following to the bottom of the file using your machine's IP address and hostname: 10.10.10.111 myhostname

!!! Perform Inital Setup euca_conf --setup

!! Start Cloud Controller (CLC), Storage Controller (SC) and Walrus !!! Start Service euca_conf --initialize service eucalyptus-cloud start

!!! Get Credentials mkdir euca-credentials cd euca-credentials euca_conf --get-credentials admin.zip unzip admin.zip source eucarc

If you get an error when you try to do this such as "index out of range", it means that the eucalyptus services aren't fully operational yet. Be patient and eventually you can get your credentials.

!!! Register Walrus, CC, and SC The component and storage flags are arbitrary names for the registered services. Change to your liking. In our specific case in CS401, Walrus and the SC are running on the same machine as the CLC, so the hostname is simply the static IP of the Morpheus server. su eucalyptus euca_conf --register-walrus --host --component walrus --partition mycloud euca_conf --register-sc --host --component storage --partition mycloud euca_conf --register-cluster --host --component cluster --partition mycloud

It is important to note that the partition name for the Storage Controller and Cluster Controller MUST BE THE SAME. If the names are different then your cloud will NOT WORK.

!! Cluster Controller !!! Start Service service eucalyptus-cc start

!! Node Controllers Run on each NC

!!! Install Packages yum install eucalyptus-nc

!!! Configure Ethernet Bridge Enter the network configuration directory cd /etc/sysconfig/network-scripts/

Open a new file called ifcfg-br0 in your text editor of choice vim ifcfg-br0

Here is an example bridge configuration. Essentially, move your configuration from ifcfg-eth0 to ifcfg-br0. Your ethernet device configuration will resemble this: DEVICE="eth0" TYPE="Ethernet" HWADDR="00:18:8B:81:AE:E4" BRIDGE="br0"

Your bridge configuration will resemble this: DEVICE="br0" TYPE="Bridge" BOOTPROTO="static" BROADCAST="10.15.255.255" DNS1="10.13.1.25" GATEWAY="10.15.1.1" IPADDR="10.15.15.13" NETMASK="255.255.0.0" ONBOOT="yes"

Restart the network service: service network restart

If all goes well your devices and bridge will come up.

!!! Register Node with CC From the CLC run: euca_conf --register-nodes="nodeip1 nodeip2 etc"

!!! Start Services service libvirtd start service eucalyptus-nc start

!! Verify Installation

!!! Verify Registered Services on CLC euca-describe-walruses euca-describe-storage-controllers euca-describe-clusters

Sample Output: WALRUS walrus          walrus          10.15.15.10                     ENABLED {} STORAGECONTROLLER      storage         storage         10.15.15.10                     ENABLED {} CLUSTER trinity        trinity         10.15.15.12                     ENABLED {}

!!! Verify Cluster is Advertising Resources euca-describe-availability-zones verbose

Sample Output (that should be updated when the cluster actually advertises resources): AVAILABILITYZONE       trinity 10.15.15.12 arn:euca:eucalyptus:trinity:cluster:trinity/ AVAILABILITYZONE       |- vm types     free / max   cpu   ram  disk AVAILABILITYZONE       |- m1.small     0002 / 0002   1    128     2 AVAILABILITYZONE       |- c1.medium    0002 / 0002   1    256     5 AVAILABILITYZONE       |- m1.large     0001 / 0001   2    512    10 AVAILABILITYZONE       |- m1.xlarge    0000 / 0000   2   1024    20 AVAILABILITYZONE       |- c1.xlarge    0000 / 0000   4   2048    20