• View
• Edit
• History
• Print

Eucalyptus Installation

Overview

These are the steps that were taken to create a functional Eucalyptus 3.1 system. Theses intstructions are for CentOS 6.1. They may or may not work on other distributions. (Definitely won't work for any non-RedHat OS)

Hardware

This guide assumes that you have 2 or more computers to use in order to build a Eucalyptus cloud. These computers must support hardware virtualization (Read the "Prepare System" section for more info). This guide will be documenting how to set up a Eucalyptus cloud using the MANAGED networking mode. In our case, this required the machine that is running the Cluster Controller (CC) to have a second Network Interface Card (NIC) so that all Node Controllers (NCs) were on their own subnet.

Prepare System

We need to do a few things to make sure that our system is ready for Eucalyptus to be installed.

Update the BIOS is one of the things we needed to do, Here is how you can do it too:

Step 1: Downloading FreeDOS to use to boot the machine and run the BIOS .EXE

wget http://www.fdos.org/bootdisks/autogen/FDOEM.144.gz Then, gunzip FDOEM.144.gz to unzip the file.

Step 2: Copy the BIOS flash utility and the BIOS image that needs to be used to upgrade, and mount it to a floppy disk image. Here is what you do in order to complete this step:

modprobe vfat

modprobe loop

mkdir /tmp/floppy

mount -t vfat -o loop FDOEM.144 /tmp/floppy

After mounting the Floppy you want to copy the EXE. that you downloaded for the BIOS:

cp DELLBIOSVERSION.exe /tmp/floppy (not actual name of the bios)

then unmount the floppy: umount /tmp/floppy

Step 3 is to burn the bootable CD which emulates a floppy device.

mkisofs -o bootcd.iso -b FDOEM.144 FDOEM.144 cdrecord -v bootcd.iso

Now you should have a bootable CD that can boot into FreeDOS where you can then run the BIOS .EXE that you have put on it.

Disable SELinux

Eucalyptus does not work with SELinux (Security Enhanced Linux). Disable it! Open the following file in your favorite text editor (I use vim):

    vim /etc/selinux/config

Change the following line:

    SELINUX=enforcing

To this:

    SELINUX=permissive

Then run:

    setenforce 0

Open Ports (or Disable Firewall)

You have a choice here. You can either open the following ports by editing iptables, or just turn off the firewall. If you want to add firewall rules to allow traffic on those ports, the easiest way is by running...

    system-config-firewall-tui

... and following the prompts to open the following ports on the machines you will be using for the CLC/CC:

1. 8443
2. 8773
3. 8774
4. 9001

Open port 8775 on all Node Controllers.

Don't feel like opening ports? Select the option to disable the firewall in `system-config-firewall-tui` and move on with your life.

Check for Hardware Virtualization

We are going to be using KVM (Kernel-based Virtual Machine) for our virtual machine instances, as is this is what RedHat is supporting now rather than Xen. KVM requires that your CPU have hardware virtualization support (Intel VT or AMD-V) in order to work.

To check for that, run the following command:

    egrep '^flags.*(vmx|svm)' /proc/cpuinfo

If there is NO output then your CPU does NOT have hardware virtualization support, and therefore cannot use KVM. If you're willing to sell your soul you may still be able to use VMWare...

Sync with NTP Time Server

The system clocks need to synchronized across all machines. Install the Network Time Protocol daemon for this. Install the ntp package if it is not already installed:

    yum install -y ntp

Sync with time server:

    ntpdate pool.ntp.org

Start the service:

    service ntpd start

Install Dependencies

Install Additional Repositories

Download the following .rpm package files More info here.

    wget http://yum.pgrpms.org/9.1/redhat/rhel-6-i386/pgdg-centos91-9.1-4.noarch.rpm
    wget http://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpm
    wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm
    wget http://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-1.el.noarch.rpm
    yum localinstall pgdg-centos91-9.1-4.noarch.rpm epel-release-6-5.noarch.rpm \
                     elrepo-release-6-4.el6.elrepo.noarch.rpm \
                     eucalyptus-nightly-release-3-1.el.noarch.rpm

Necessary Manual Edit

Note: DO THIS IF AND ONLY IF YOU ARE RUNNING A 32 BIT VERSION OF CENTOS.

It is HIGHLY recommended that you use a 64 bit OS, but making this change allows you to get some .noarch packages from the Eucalyptus repository. Otherwise, the package list will be 404 Not Found. Edit /etc/yum.repos.d/eucalyptus-nightly-release.repo This is the file for the Eucalyptus 3 development repositories. The x86_64 repos had to be used because the i686/i386 repositories gave a 404 error.

    [euca-3-devel]
    name=euca-3-devel
    baseurl=http://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/$releasever/x86_64/
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLY
    gpgcheck=1
    enabled=0

    [euca-3-deps]
    name=euca-3-deps
    baseurl=http://downloads.eucalyptus.com/devel/packages/3-devel/centos/$releasever/x86_64/
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLY
    gpgcheck=1
    enabled=1

The changes that should be made are in bold.

Install Packages

Source: http://agrimmsreality.blogspot.com/2012/01/building-eucalyptus-3-devel.html

    yum install bzr python-boto euca2ools libvirt-devel openssl-devel gcc ant ant-nodeps \
    java-1.6.0-openjdk-devel curl-devel libxslt-devel xalan-j2-xsltc wsdl4j \
    backport-util-concurrent httpd postgresql91-server libvirt PyGreSQL make \
    openssh-clients scsi-target-utils qemu-kvm axis2-codegen axis2-adb-codegen \
    httpd httpd-devel rampartc rampartc-devel axis2c axis2c-devel axis2

Create eucalyptus User

We need to add a eucalyptus user for the Eucalyptus services to run as. It is added to the kvm group so machines that are node controllers can manipulate instances.

    useradd eucalyptus 
    passwd eucalyptus

Modify WSDL2C.sh

Open /usr/lib64/axis2c/bin/tools/wsdl2c/WSDL2C.sh in a text editor. Erase the existing lines and add the following:

    #!/bin/sh

    java -classpath $(build-classpath axis2/codegen axis2/kernel axis2/adb \
     axis2/adb-codegen wsdl4j commons-logging xalan-j2 xsltc \
     backport-util-concurrent ws-commons-XmlSchema ws-commons-neethi \
     ws-commons-axiom annogen ) org.apache.axis2.wsdl.WSDL2C $*

Compile Dependencies

DON'T ACTUALLY DO THIS ON A 64 BIT MACHINE. The eucalyptus repository that was installed in a previous step has all of the packages you need. This is here for historical purposes and will probably be deleted soon. Note: Compilation only needs to happen on one machine. Once compiled, the binaries can be copied to the other servers

This section is copied pretty much verbatim from the Eucalyptus 2.0 Installation Instructions.

Download Source Code

Download and extract tarball (2.0.3 dependencies work for 3.0 as far as I can tell)

    wget http://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-deps.tar.gz
    tar -zxf eucalyptus-2.0.3-src-deps.tar.gz

Set shell variable for install location Dave's completely objective, non-biased Note: Installing to /opt is really ugly, but this what the Eucalyptus install guide uses, so I will too.

    export EUCALYPTUS=/opt/eucalyptus

Create directory for dependency binaries

    mkdir -p $EUCALYPTUS/packages/

Change directory to where the dependency sources were extracted Your path will probably be different

    cd ~/Downloads/eucalyptus-src-deps/

Install Axis2

Extract and copy Axis2 No compilation here. Nice and easy.

    tar -zxf axis2-1.4.tgz
    mv axis2-1.4 $EUCALYPTUS/packages/

Install Axis2C

    export APACHE_INCLUDES=/usr/include/httpd/
    export APR_INCLUDES=/usr/include/apr-1/
    export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.6.0

    tar zxf axis2c-src-1.6.0.tar.gz
    cd axis2c-src-1.6.0
    CFLAGS="-w" ./configure --prefix=${AXIS2C_HOME} --with-apache2=$APACHE_INCLUDES --with-apr=$APR_INCLUDES --enable-multi-thread=no
    make
    make install

Install Rampart/C

    export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib:$LD_LIBRARY_PATH

    tar zxf rampartc-src-1.3.0-0euca2.tar.gz
    cd rampartc-src-1.3.0
    ./configure --prefix=${AXIS2C_HOME} --enable-static=no --with-axis2=${AXIS2C_HOME}/include/axis2-1.6.0
    make
    make install

Edit Rampart/C Configuration

Change the following in $AXIS2C_HOME/axis2.xml. In the 'inflow' section, change:

        <!--phase name="Security"/-->

to

        <phase name="Security"/>

In the 'outflow' section, change:

        <!--phase name="Security"/-->

to

        <phase name="Security"/>

Compiling Eucalyptus

Download Source Code

Fetch the latest source from launchpad

    bzr branch lp:eucalyptus

Enter the source directory

    cd eucalyptus

Compile

Set shell variables for Java environment:

    export JAVA_HOME="/usr/lib/jvm/java-openjdk/"
    export JAVA="$JAVA_HOME/jre/bin/java"

Compile Eucalyptus

    ./configure --with-axis2c=/usr/lib64/axis2c/ --with-apache2-module-dir=/usr/lib64/httpd/modules/
    make
    make install

Install Eucalyptus on Other Machines

Based heavily upon this wonderful blog post.

Copy Installation to All Machines

Repeat for each machine.

    rsync -a $EUCALYPTUS/ root@<HOST>:$EUCALYPTUS/  (<HOST> being the IP of the node you are copying to)

Set environment variables

These are needed pretty much everywhere. A good idea would be to place these in the ~/.bashrc files for root and eucalyptus users. You WILL have commands fail if you don't set these.

    export EUCALYPTUS=/opt/eucalyptus
    export PATH=$PATH:$EUCALYPTUS/usr/sbin

Install Packages

Follow the "Installing Additional Packages" and "Installing Dependencies" instructions at the top of this page to install needed packages onto the node

Configure

Edit /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf Find the following configuration variables and set the values accordingly

    EUCALYPTUS="/opt/eucalyptus"
    HYPERVISOR="kvm"
    USE_VIRTIO_DISK="1"
    USE_VIRTIO_NET="1"
    INSTANCE_PATH="/opt/eucalyptus/instances"
    VNET_BRIDGE="virbr0"

Register Startup Scripts

Create symbolic links to the daemons so that the `service` command will work with them

    ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud
    ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc
    ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc

Edit Hosts

A common problem that occurs is that the `euca_conf --initialize` command will fail. This usually happens because the system hostname isn't in the /etc/hosts file.

Edit hosts file:

    vim /etc/hosts

Add something like the following to the bottom of the file using your machine's IP address and hostname:

    10.10.10.111 myhostname

Perform Inital Setup

    euca_conf --setup  

Start Cloud Controller (CLC), Storage Controller (SC) and Walrus

Start Service

    euca_conf --initialize
    service eucalyptus-cloud start

Get Credentials

    mkdir euca-credentials
    cd euca-credentials
    euca_conf --get-credentials admin.zip
    unzip admin.zip
    source eucarc

If you get an error when you try to do this such as "index out of range", it means that the eucalyptus services aren't fully operational yet. Be patient and eventually you can get your credentials.

Register Walrus, CC, and SC

The component and storage flags are arbitrary names for the registered services. Change to your liking. In our specific case in CS401, Walrus and the SC are running on the same machine as the CLC, so the hostname is simply the static IP of the Morpheus server.

    su eucalyptus
    euca_conf --register-walrus --host <hostname> --component walrus --partition mycloud
    euca_conf --register-sc --host <hostname> --component storage --partition mycloud
    euca_conf --register-cluster --host <hostname> --component cluster --partition mycloud

It is important to note that the partition name for the Storage Controller and Cluster Controller MUST BE THE SAME. If the names are different then your cloud will NOT WORK.

Cluster Controller

Start Service

    service eucalyptus-cc start

Node Controllers

Run on each NC

Install Packages

    yum install eucalyptus-nc

Configure Ethernet Bridge

Enter the network configuration directory

    cd /etc/sysconfig/network-scripts/

Open a new file called ifcfg-br0 in your text editor of choice

    vim ifcfg-br0

Here is an example bridge configuration. Essentially, move your configuration from ifcfg-eth0 to ifcfg-br0. Your ethernet device configuration will resemble this:

    DEVICE="eth0"
    TYPE="Ethernet"
    HWADDR="00:18:8B:81:AE:E4"
    BRIDGE="br0"

Your bridge configuration will resemble this:

    DEVICE="br0"
    TYPE="Bridge"
    BOOTPROTO="static"
    BROADCAST="10.15.255.255"
    DNS1="10.13.1.25"
    GATEWAY="10.15.1.1"
    IPADDR="10.15.15.13"
    NETMASK="255.255.0.0"
    ONBOOT="yes"

Restart the network service:

    service network restart

If all goes well your devices and bridge will come up.

Register Node with CC

From the CLC run:

    euca_conf --register-nodes="nodeip1 nodeip2 etc"

Start Services

    service libvirtd start
    service eucalyptus-nc start

Verify Installation

Verify Registered Services on CLC

    euca-describe-walruses
    euca-describe-storage-controllers
    euca-describe-clusters

Sample Output:

    WALRUS  walrus          walrus          10.15.15.10                     ENABLED {}
    STORAGECONTROLLER       storage         storage         10.15.15.10                     ENABLED {}
    CLUSTER trinity         trinity         10.15.15.12                     ENABLED {}

Verify Cluster is Advertising Resources

    euca-describe-availability-zones verbose

Sample Output (that should be updated when the cluster actually advertises resources):

    AVAILABILITYZONE        trinity 10.15.15.12 arn:euca:eucalyptus:trinity:cluster:trinity/
    AVAILABILITYZONE        |- vm types     free / max   cpu   ram  disk
    AVAILABILITYZONE        |- m1.small     0002 / 0002   1    128     2
    AVAILABILITYZONE        |- c1.medium    0002 / 0002   1    256     5
    AVAILABILITYZONE        |- m1.large     0001 / 0001   2    512    10
    AVAILABILITYZONE        |- m1.xlarge    0000 / 0000   2   1024    20
    AVAILABILITYZONE        |- c1.xlarge    0000 / 0000   4   2048    20