Apache CloudStack

Version 4.1.1 Release Notes

Apache CloudStack

Legal Notice

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Abstract

Release notes for the Apache CloudStack 4.1.1 release.

Preface

1. Document Conventions

This manual uses several conventions to highlight certain words and phrases and draw attention to specific pieces of information.

In PDF and paper editions, this manual uses typefaces drawn from the Liberation Fonts set. The Liberation Fonts set is also used in HTML editions if the set is installed on your system. If not, alternative but equivalent typefaces are displayed. Note: Red Hat Enterprise Linux 5 and later includes the Liberation Fonts set by default.

1.1. Typographic Conventions

Four typographic conventions are used to call attention to specific words and phrases. These conventions, and the circumstances they apply to, are as follows.

Mono-spaced Bold

Used to highlight system input, including shell commands, file names and paths. Also used to highlight keycaps and key combinations. For example:

To see the contents of the file my_next_bestselling_novel in your current working directory, enter the cat my_next_bestselling_novel command at the shell prompt and press Enter to execute the command.

The above includes a file name, a shell command and a keycap, all presented in mono-spaced bold and all distinguishable thanks to context.

Key combinations can be distinguished from keycaps by the hyphen connecting each part of a key combination. For example:

Press Enter to execute the command.
Press Ctrl+Alt+F2 to switch to the first virtual terminal. Press Ctrl+Alt+F1 to return to your X-Windows session.

The first paragraph highlights the particular keycap to press. The second highlights two key combinations (each a set of three keycaps with each set pressed simultaneously).

If source code is discussed, class names, methods, functions, variable names and returned values mentioned within a paragraph will be presented as above, in mono-spaced bold. For example:

File-related classes include filesystem for file systems, file for files, and dir for directories. Each class has its own associated set of permissions.

Proportional Bold

This denotes words or phrases encountered on a system, including application names; dialog box text; labeled buttons; check-box and radio button labels; menu titles and sub-menu titles. For example:

Choose System → Preferences → Mouse from the main menu bar to launch Mouse Preferences. In the Buttons tab, click the Left-handed mouse check box and click Close to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand).
To insert a special character into a gedit file, choose Applications → Accessories → Character Map from the main menu bar. Next, choose Search → Find… from the Character Map menu bar, type the name of the character in the Search field and click Next. The character you sought will be highlighted in the Character Table. Double-click this highlighted character to place it in the Text to copy field and then click the Copy button. Now switch back to your document and choose Edit → Paste from the gedit menu bar.

The above text includes application names; system-wide menu names and items; application-specific menu names; and buttons and text found within a GUI interface, all presented in proportional bold and all distinguishable by context.

Mono-spaced Bold Italic or Proportional Bold Italic

Whether mono-spaced bold or proportional bold, the addition of italics indicates replaceable or variable text. Italics denotes text you do not input literally or displayed text that changes depending on circumstance. For example:

To connect to a remote machine using ssh, type ssh username@domain.name at a shell prompt. If the remote machine is example.com and your username on that machine is john, type ssh john@example.com.
The mount -o remount file-system command remounts the named file system. For example, to remount the /home file system, the command is mount -o remount /home.
To see the version of a currently installed package, use the rpm -q package command. It will return a result as follows: package-version-release.

Note the words in bold italics above — username, domain.name, file-system, package, version and release. Each word is a placeholder, either for text you enter when issuing a command or for text displayed by the system.

Aside from standard usage for presenting the title of a work, italics denotes the first use of a new and important term. For example:

Publican is a DocBook publishing system.

1.2. Pull-quote Conventions

Terminal output and source code listings are set off visually from the surrounding text.

Output sent to a terminal is set in mono-spaced roman and presented thus:

books        Desktop   documentation  drafts  mss    photos   stuff  svn
books_tests  Desktop1  downloads      images  notes  scripts  svgs

Source-code listings are also set in mono-spaced roman but add syntax highlighting as follows:

package org.jboss.book.jca.ex1;

import javax.naming.InitialContext;

public class ExClient
{
   public static void main(String args[]) 
       throws Exception
   {
      InitialContext iniCtx = new InitialContext();
      Object         ref    = iniCtx.lookup("EchoBean");
      EchoHome       home   = (EchoHome) ref;
      Echo           echo   = home.create();

      System.out.println("Created Echo");

      System.out.println("Echo.echo('Hello') = " + echo.echo("Hello"));
   }
}

1.3. Notes and Warnings

Finally, we use three visual styles to draw attention to information that might otherwise be overlooked.

Note

Notes are tips, shortcuts or alternative approaches to the task at hand. Ignoring a note should have no negative consequences, but you might miss out on a trick that makes your life easier.

Important

Important boxes detail things that are easily missed: configuration changes that only apply to the current session, or services that need restarting before an update will apply. Ignoring a box labeled 'Important' will not cause data loss but may cause irritation and frustration.

Warning

Warnings should not be ignored. Ignoring warnings will most likely cause data loss.

2. Submitting Feedback and Getting Help

If you find a typographical error in this manual, or if you have thought of a way to make this manual better, we would love to hear from you! Please submit a bug: https://issues.apache.org/jira/browse/CLOUDSTACK against the component Doc.

If you have a suggestion for improving the documentation, try to be as specific as possible when describing it. If you have found an error, please include the section number and some of the surrounding text so we can find it easily.

Better yet, feel free to submit a patch if you would like to enhance the documentation. Our documentation is, along with the rest of the CloudStack source code, kept in the project's git repository.

The most efficient way to get help with CloudStack is to ask on the mailing lists.

The Apache CloudStack project has mailing lists for users and developers. These are the official channels of communication for the project and are the best way to get answers about using and contributing to CloudStack. It's a good idea to subscribe to the users@cloudstack.apache.org mailing list if you've deployed or are deploying CloudStack into production, and even for test deployments.

The CloudStack developer's mailing list (dev@cloudstack.apache.org) is for discussions about CloudStack development, and is the best list for discussing possible bugs in CloudStack. Anyone contributing to CloudStack should be on this mailing list.

To posts to the lists, you'll need to be subscribed. See the CloudStack Web site for instructions.

Chapter 1. Welcome to CloudStack 4.1

Welcome to the 4.1.1 release of CloudStack, the first bug fix release of CloudStack in the 4.1.x line.

This document contains information specific to this release of CloudStack, including upgrade instructions from prior releases and issues fixed in the release. For installation instructions, please see the Installation Guide. For usage and administration instructions, please see the CloudStack Administrator's Guide. Developers and users who wish to work with the API will find instruction in the CloudStack API Developer's Guide

If you find any errors or problems in this guide, please see Section 2, “Submitting Feedback and Getting Help”. We hope you enjoy working with CloudStack!

Chapter 2. Compatibility Matrix for 4.1.1

2.1. Supported Operating Systems
2.2. Supported Hypervisors
2.3. Supported Browsers
2.4. External Devices

CloudStack is tested against certain operating systems, hypervisors, and other components to ensure that it works on specific platforms. It may work well on other platforms, but the platforms listed below are the ones we specifically test against and are more likely to be able to help troubleshoot if you run into any issues.

2.1. Supported Operating Systems

This section lists the operating systems that are supported for running CloudStack's Management Server.

Note that we test against specific versions of the OSes, so compatibility with CentOS 6.3 may not indicate compatibility with CentOS 6.2, etc.

CentOS 6.3
Red Hat Enterprise Linux 6.3
Ubuntu 12.04 LTS

2.2. Supported Hypervisors

CloudStack supports three hypervisor families, Xen with XAPI, KVM, and VMware with vSphere.

CentOS 6.2 with KVM
Red Hat Enterprise Linux 6.2 with KVM
XenServer 6.0.2 (with Hotfix)
XenServer 6.1
VMware vSphere/Vcenter 5.1

Bare Metal Support

Bare metal support is not present in this release.

2.3. Supported Browsers

The CloudStack Web-based UI should be compatible with any modern browser, but it's possible that some browsers will not render portions of the UI reliably, depending on their support of Web standards. For best results, we recommend one of the following browsers.

Internet Explorer 8
Firefox 10+
Chrome
Safari

Note that it's difficult to confirm specific browser versions for Firefox and Google Chrome, given the speed of their update cycle.

2.4. External Devices

The following external devices are supported in CloudStack 4.1.1.

F5: 10.1.10 (Build 3341.1084)
SRX model srx100b: Must be 10.3 or higher -10.4R7.5
Netscaler VPX 9.3, 10.0(Build 54.7.nc and 54.161)
Netscaler MPX 10
Netscaler SDX 10

Chapter 3. Version 4.1.1

3.1. Issues Fixed in 4.1.1
3.2. Known Issues in 4.1.1

3.1. Issues Fixed in 4.1.1

CloudStack uses Jira to track its issues. All bugs for 4.1.1 have been tracked in Jira, and have a standard naming convention of "CLOUDSTACK-NNNN" where "NNNN" is the issue number.

This section includes a summary of known issues against 4.1.0 that were fixed in 4.1.1. Approximately 36 bugs were resolved or closed in the 4.1.1 cycle.

Defect	Description
CLOUDSTACK-531	Potential disaster with template-cleanup enabled!
CLOUDSTACK-2523	Object_Store_Refactor - Recurring Snapshots are failing because of NullPointerException.
CLOUDSTACK-2758	cloudstack-management does not start from RPM packages
CLOUDSTACK-2905	Recurring Snapshots are failing becuase of NullPointerException.
CLOUDSTACK-3005	Upgrade from 2.2.14 to 4.1.0 failed with "Storage volume not found: no storage vol with matching name"
CLOUDSTACK-3015	VPC virtual router lists deleted nics
CLOUDSTACK-3097	Installation docs have the wrong package names
CLOUDSTACK-3278	In non-OSS build, jetty:run does not upgrade the database to 4.1.0 and 4.1.1
CLOUDSTACK-1353	KVM 6.3 snapshot Scheduling snapshot failed due to java.lang.NullPointerException
CLOUDSTACK-3004	[script] ssvm_check remove the duplicate file from consoleproxy and include the script from secondary storage folder while packing iso
CLOUDSTACK-3080	listVirtualMachines tag search refinement no longer working
CLOUDSTACK-3179	[DOC] Service cloud-management should be changed to service cloudstack-management
CLOUDSTACK-3180	[DOC] Path /usr/lib64/cloud/common/ for CentOS should be changed to /usr/share/cloudstack-common
CLOUDSTACK-3182	[DOC] Path /etc/cloud should be changed to /etc/cloudstack
CLOUDSTACK-3440	Action events are not published due to conflict of Eventutils and ActionEventUtils files
CLOUDSTACK-3456	We need to proactively close XAPI initiated console session from console proxy to avoid Deny of Service behave to XenServer host
CLOUDSTACK-3540	edithosts.sh script may delete the active dhcphosts.txt entry
CLOUDSTACK-305	AWS APi - "Rolling back the transaction" seen in management server logs , everytime a soap call is made.
CLOUDSTACK-308	ec2-describe-instances - Instance type is always retuned as "m1.small"
CLOUDSTACK-1475	RegisterISO error after Update SSL Certificate
CLOUDSTACK-1759	PF test cases failing with error "unexpected keyword argument 'keyPairFileLocation'"
CLOUDSTACK-1760	[Automation] Router test cases failed while checking dnsmasq service status
CLOUDSTACK-2551	Update Quick Install Guide
CLOUDSTACK-2893	The Agent attempts to re-create a already existing Libvirt Storage pool when creating a volume
CLOUDSTACK-2936	Disallow special characters on all text fields
CLOUDSTACK-3213	ResourceStateEvents for user VM are generated with entity type as Network
CLOUDSTACK-3328	cloudstack-agent RPM package doesn't require qemu-kvm
CLOUDSTACK-3407	network remains in 'allocated' if nic is added to it as first action
CLOUDSTACK-3408	userdata not available from all nics
CLOUDSTACK-3409	Security groups get clean up if VM is not in a running state
CLOUDSTACK-986	Add Ubuntu instructions to 'Using the Management Server As the NFS Server'
CLOUDSTACK-2338	Add example of how to sign an API request
CLOUDSTACK-2339	Add libcloud example
CLOUDSTACK-2859	configurations for secondary external DNS appear to be wrong (4.4.4.4)
CLOUDSTACK-2902	baseurl incorrect in installation guide

3.2. Known Issues in 4.1.1

Issue ID	Description
CLOUDSTACK-77	console proxy display issues
CLOUDSTACK-79	CloudStack 3.0.4: firewall rules not restored on KVM host
CLOUDSTACK-83	hitting exception when trying to take two consecutive snapshot on same volume
CLOUDSTACK-107	Network domain guest suffix is not getting programmed as part of hostnames on Guest VMs that are part of Isolated and Shared Guest Networks
CLOUDSTACK-124	NetworkGarbageCollector not cleaning up networks
CLOUDSTACK-155	HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
CLOUDSTACK-216	Templates' Source Location Information present on "template_host_ref" and "vm_template" tables should refer to Apache System Template Locations
CLOUDSTACK-231	Tag creation using special charecters
CLOUDSTACK-237	StopVMCommand reported success in spite of failing to stop a VM which got stuck during installation from an ISO
CLOUDSTACK-238	vpn:fail to connect to vpnserver using non-sourceNAT IP
CLOUDSTACK-242	haproxy listens on all interfaces on VR
CLOUDSTACK-243	On management server, security for remote JMX connections is disabled
CLOUDSTACK-244	RPC port on SSVM is open on all interfaces
CLOUDSTACK-245	VPC ACLs are not stored and programmed consistently
CLOUDSTACK-252	UpdateNetwork Operation on a guest network that is currently using Virtual Router for Lb services to a network offering that uses "F5" for Lb services Fails due to MySQLIntegrityConstraintViolationException.
CLOUDSTACK-255	Null pointer exception while creating portforwarding rule after performing UpdateNetworkCmd
CLOUDSTACK-270	Ui should not ask for a vlan range if the physical network isolation type is not VLAN
CLOUDSTACK-272	Delete failure message for network with a VM is not informative
CLOUDSTACK-280	Exception thrown on going to Step 5 of Add VM Instance - CloudRuntimeException: Tags are not defined for physical network in the zone id=1
CLOUDSTACK-282	Virtual Routers do not properly resolve DNS SRV Records
CLOUDSTACK-285	Snapshots fail to respect retention period if schedule is deleted and re-created.
CLOUDSTACK-295	Add instructions to modify limits.conf in installation directions
CLOUDSTACK-298	putting host in maintenance mode while creating snapshot ,host resorce state stuck in "ErrorInMaintenance mode" and snapshot creation fail Unable to migrate due to Requested operation is not valid: cannot migrate domain with 1 snapshots
CLOUDSTACK-300	Creation of compute offering allow combination of local storage + HA
CLOUDSTACK-310	Failed to add host - Plugin error
CLOUDSTACK-315	Infrastructure view does not show capacity values
CLOUDSTACK-324	Cannot edit default security group rules, default security group blocks all inbound traffic.
CLOUDSTACK-338	Unique Names of Disk and Service Offerings in the database are prefixed with "Cloud.com" String
CLOUDSTACK-360	System VM template isn't copied from Secondary Storage to XenServer's local SR
CLOUDSTACK-371	When naming physical networks in the zone wizard, special characters like () break the wizard
CLOUDSTACK-375	Unable to delete physical network - because there are other networks attached
CLOUDSTACK-381	Install Guide: Section 2.5: Notes that you can skip secondary storage if you are using local disks.
CLOUDSTACK-382	Install Guide: Network Type Descriptions
CLOUDSTACK-410	vnc_listen not configured in qemu.conf for Ubuntu KVM host
CLOUDSTACK-425	Check image type is qcow2 before actually installing
CLOUDSTACK-429	Agent rebalancing is broken
CLOUDSTACK-433	IP addresses can be left assigned and orphaned when static NAT assign fails
CLOUDSTACK-440	create networks in advanced zone with out VLAN isolation
CLOUDSTACK-458	xen:snapshots:Storage gc fail to clean the failed snapshot images from secondarystorage
CLOUDSTACK-463	mvn does not inject agent version into .JAR files.
CLOUDSTACK-468	Difficult to find API reference when there is no link to it from the Documentation landing page
CLOUDSTACK-469	CloudStack Documentation Landing Page has Alignment Issues
CLOUDSTACK-470	Add vm does not list all vms when creating port forwards
CLOUDSTACK-482	Installation Guide Doc Error Section 4.5.7
CLOUDSTACK-486	Clicking UI notifications for System VM or Virtual Router opens Instances page
CLOUDSTACK-489	Document how to use userdata feature for user vms
CLOUDSTACK-506	[UI] Unable to execute API command liststoragepools
CLOUDSTACK-519	Cloudstack development environment on Windows using Cygwin references 'sudo' which is not available in Cygwin
CLOUDSTACK-525	Cloudstack Redundant Virtual Router issues both MASTER.
CLOUDSTACK-526	Cloudstack Redundant Virtual Router issues with Port Forwarding
CLOUDSTACK-532	Storage-template-cleanup corrupts templates
CLOUDSTACK-535	Virtual Router DNS is restricted to UDP only
CLOUDSTACK-554	Cloudstack should virtualized 32bits CPUs for 32bits templates
CLOUDSTACK-561	createNetwork can block for long periods causing clients to time out when called at certain times
CLOUDSTACK-562	Unable to download DATA volume on CS4 and vSphere 5
CLOUDSTACK-563	Unable to create guest network in UI for Advanced Zone
CLOUDSTACK-568	Source template id is recorded incorrectly.
CLOUDSTACK-575	Cannot change memory size of the console.ram.size, ssvm.ram.size using web console
CLOUDSTACK-582	Cannot perform password reset for Windows instances on VMWare
CLOUDSTACK-586	exportfs -a gives 'no_subtree_check warning' when following the Installation guide
CLOUDSTACK-589	Juniper SRX does not depend on f5 iControl jar
CLOUDSTACK-597	Management Server Restart loses hosts
CLOUDSTACK-626	guest_os table does not have latest distro
CLOUDSTACK-649	KVM storage gc does not delete the invalid snapshots from physical storage
CLOUDSTACK-668	Secondary storage host always remains in Alert status
CLOUDSTACK-688	UI Russian language
CLOUDSTACK-696	Networks with same name and VLAN are allowed in the same Zone
CLOUDSTACK-722	Using a certificate chain for the Console Proxy is not documented
CLOUDSTACK-732	Add back KVM snapshot support
CLOUDSTACK-904	CloudStack export the CPU as a socket not core
CLOUDSTACK-969	api: zone response lists vlan in it as "vlan range of zone" but the vlan belongs to physical network
CLOUDSTACK-970	when a template is deleted and then copied over again , it is still marked as "Removed" in template_zone_ref table.
CLOUDSTACK-989	marvin: jsonHelper deserialization results in unfilled attributes
CLOUDSTACK-992	Template creations dies after 2h for no reason
CLOUDSTACK-994	Feature Request: Support VmWare Storage Pool
CLOUDSTACK-1007	Not able to delete Shared network because of not being able to stop the router.
CLOUDSTACK-1018	IPtables manipulation is EL-specific and doesn't cover Ubuntu
CLOUDSTACK-1025	If an ISO is deleted there is no check if the ISO is actually attached to a guest or not.
CLOUDSTACK-1048	Launching AutoScale: Documentation Flow is Incorrect
CLOUDSTACK-1053	Start systemvm
CLOUDSTACK-1069	Document workaround for: CS and LDAP user validation can't happen simultaneously
CLOUDSTACK-1075	create an utility in Marvin to support nsnitro
CLOUDSTACK-1089	API Problem with listAsyncJobs
CLOUDSTACK-1090	VM unable to obtain guest IP from a virtual router running on another XCP/XenServer hypervisor (Cloudstack Advanced Zone Setup)
CLOUDSTACK-1167	arptables rules are not destroyed after vm is stopped in XenServer in basic zone
CLOUDSTACK-1177	CloudStack always reports 'invalid username or password' when using IE10
CLOUDSTACK-1194	Bug with the web interface (re isolation method)
CLOUDSTACK-1195	Wrong isolation method offered for KVM hypervisor
CLOUDSTACK-1197	Basic zone DNS domain not offered by VR
CLOUDSTACK-1199	External DNS not being added to reslov.conf
CLOUDSTACK-1225	Storage System vm id keeps on increasing
CLOUDSTACK-1236	Warning while adding Xen 6.1 host [Unable to create local link network]
CLOUDSTACK-1245	Not able to install Debian based PV Instances from ISO,only HVM instances is possible.
CLOUDSTACK-1249	CloudStack Service restart has inconsistent names
CLOUDSTACK-1259	NPE at service shutdown
CLOUDSTACK-1282	UI - Tooltip - Missing tooltip for Projects page
CLOUDSTACK-1283	UI - Tooltip - Need a clean way to differentiate between Alt Tag's and Tooltips.
CLOUDSTACK-1284	UI - Tooltip - Inconsistent tool tip rendering among installation wizards
CLOUDSTACK-1285	UI -Tooltip - Lengthy and inconsistent description among toolips
CLOUDSTACK-1302	Add per storage setting for cache="none/writeback/writethrough" options for VMs on KVM hypervisor
CLOUDSTACK-1304	"mvn -pl :cloud-client-ui jetty:run" strips permission of files in script/
CLOUDSTACK-1306	Better Error message when trying to deploy Vm by passing static Ipv4 addresses that are assigned to another VM/IP4 address is outside the iprange.
CLOUDSTACK-1309	Large guest subnets downgrade performance
CLOUDSTACK-1316	Documentation update for "cloud-set-guest-password" needed
CLOUDSTACK-1327	Cloudstack allows users to import huge templates from unauthorised URLs
CLOUDSTACK-1358	Clarify format of url field of the createStoragePool API command
CLOUDSTACK-1359	Clarify what we mean by GB in CloudStack documentation
CLOUDSTACK-1389	Interactive Password Prompts during Management Server Startup
CLOUDSTACK-1393	Install scripts shows wrong path to copy vhd-util
CLOUDSTACK-1408	Error creating new instance with sufficient resources!
CLOUDSTACK-1412	listUsageRecords will loop data if requested page is beyond what is available
CLOUDSTACK-1413	Need something to concretely identify the version of the code in a particular build
CLOUDSTACK-1424	Failed with NPE while deleting account when there are snapshots created for this account instances
CLOUDSTACK-1432	Inconsistent field names in "Add Cluster" dialog
CLOUDSTACK-1458	Add Zone wizard UI gets visually broken if additional browser toolbars are used.
CLOUDSTACK-1460	UI : List Storage table has unused column name Actions
CLOUDSTACK-1492	Missing Guide: Allocator Implementation Guide
CLOUDSTACK-1497	Alien VM's are deleted on migration by xenserver.
CLOUDSTACK-1500	Unable to connect to instances via View console
CLOUDSTACK-1504	VMWare is not using the VLAN tag for storage port-group
CLOUDSTACK-1605	No Documentation on Network Throttling
CLOUDSTACK-1632	Mistakes in authorizeSecurityGroup* API docs
CLOUDSTACK-1654	Management and Host installed on same machine, Management cannot get ID.
CLOUDSTACK-1660	CloudStack 4.1 RPM install Warning
CLOUDSTACK-1702	Error messages returning numeric ID's in some places instead of UUIDs
CLOUDSTACK-1703	Management Server access error: "Error while decrypting..."
CLOUDSTACK-1712	Documentation for Creating a Template doesn't match what's presented in management server
CLOUDSTACK-1717	AWS Regions - Local region entry that gets added by default should not include "/api" for its end_point.Also the endpoint should have the actual hostname instead of localhost.
CLOUDSTACK-1743	No Section on About Password and Key Encryption Though Multiple References Appear in the Install Guide
CLOUDSTACK-1775	Events related User/Domain/Account are not being generated expect for USER-DISABLE,DOMAIN-DELETE and ACCOUNT.DISABLE event.
CLOUDSTACK-1780	Client UI: i18N files (messages_xx_XX.properties) needs to be convert in ASCII with unicode char \uxxxx (and perhaps keys needs to be in alphabetical order)
CLOUDSTACK-1783	Alteration of some messages i18n files (ja, zh_CN ...) since last commit with transifex export files
CLOUDSTACK-1790	Lock down ports for all system VMs
CLOUDSTACK-1801	Upgrade from 3.0.2 to 4.0.1 Fails
CLOUDSTACK-1807	CS4 AWS S3 support - List all buckets AWS API doesnot return correct response if not buckets are created on NFS mount
CLOUDSTACK-1819	AWS Regions - Issues seen when trying to move a zone from 1 region to another.
CLOUDSTACK-1821	AWS S3 API -Get bucket by name - ACLs do not give user the required permission
CLOUDSTACK-1835	VMWare tries to use vSwitch0 when it shouldn't
CLOUDSTACK-1868	GetVmStatsCommand throws NullPointerException with VMWare
CLOUDSTACK-1872	Do not accept OVAs with multiple VMDKs for uploaded volumes or templates
CLOUDSTACK-1885	Broken testcases in 4.1
CLOUDSTACK-1896	S3-backed NFS secondary storage uses Db-based lock when DB is not available
CLOUDSTACK-1899	SRX firewall external devices - static NAT does not function
CLOUDSTACK-1932	AutoScale UI documentation doesn't mention the option appears only for NetScaler
CLOUDSTACK-1965	15.8. External Firewalls and Load Balancers Section is Incomplete
CLOUDSTACK-1967	Maintenance mode not fully working with VMWare
CLOUDSTACK-1969	Ubuntu fresh Install- SystemIntegrityChecker looking for "schema-40to410.sql" wrong location and failed to start MS
CLOUDSTACK-1970	Ubuntu - "cloudstack-setup-management" not available in "/usr/bin"
CLOUDSTACK-1986	Key translation fails for the Japanese keyboard keys ¥_,\ \|, Muhenkan, Henkan, Hiragana/Katakana, Kanji Key and Caps Loc
CLOUDSTACK-1992	Cannot deploy VM from template on VMWare, no issues when deploying from ISO
CLOUDSTACK-1996	Storage tags ignored when using VMWare
CLOUDSTACK-2015	CS3.0.2 VR is pending in starting state，when VR is recreated.
CLOUDSTACK-2018	[DOC] How system.vm passwords can be randomized
CLOUDSTACK-2024	cloudstack-setup-management with https not works (incorrect path and missing keystore file)
CLOUDSTACK-2025	Failed VM creation on Xen Server does not release shared IP address
CLOUDSTACK-2047	Cloudstack Management Server host key is set to localhost
CLOUDSTACK-2050	encode.api.response in global settings when set to true will create garbled % output int he gui and json output
CLOUDSTACK-2053	[BUG] userdata.sh on router VM fixes for if else logic and missing meta-data
CLOUDSTACK-2072	Virtual Routers Failover not working
CLOUDSTACK-2075	List capacity error causes UI to display status overlay
CLOUDSTACK-2183	Database upgrade from 3.0.2 to 4.0.1 failed
CLOUDSTACK-2203	SSVM does not redeploy after destroyed
CLOUDSTACK-2213	russian language select failure
CLOUDSTACK-2215	SSVM does not use allocated storage ip range
CLOUDSTACK-2216	CloudStack Manager sends wrong validBackupUUIDs to Secondary Storage VM when clean up snapshot backup. This causes the snapshots backup not garbage collected correctly and gets the secondary storage filled up.
CLOUDSTACK-2217	Snapshots backups are not deleted according to CleanupSnapshotBackupCommand. Those snapshots backups that should be garbage collected but not get deleted can fill up the secondary storage.
CLOUDSTACK-2224	"filterwin2k" should not be set in dnsmasq.conf
CLOUDSTACK-2299	Stale DHCP leases block IP reallocation
CLOUDSTACK-2300	Unexpected deletion of Snapshots backups
CLOUDSTACK-2325	incomplete sentence in installation guide
CLOUDSTACK-2344	not enough free memory on one host of the cluster
CLOUDSTACK-2353	Figure out how spring can allow for inheritable component contexts
CLOUDSTACK-2383	"Entity already exists" error at automatic virtual router VM creation.
CLOUDSTACK-2388	HA "Host Allocator returning 0 suitable hosts" but starting the VM manually finds one.
CLOUDSTACK-2446	System Capacity Zone total memory calculation is wrong
CLOUDSTACK-2448	XenServer Guest OS Corrections
CLOUDSTACK-2455	Cannot add a Debian host to a KVM cluster
CLOUDSTACK-2530	fix npe if no network isolation methods
CLOUDSTACK-2532	remove bogus self assign to parent
CLOUDSTACK-2535	Cleanup port-profiles that gets created on Nexus switch as part of network cleanup
CLOUDSTACK-2574	Release note pdf link
CLOUDSTACK-2608	Vmware network labels are ignored when creating a Zone using advance networking
CLOUDSTACK-2610	Unable to add host due to missing setiptables plugin
CLOUDSTACK-2616	com.mysql.jdbc.exceptions.jdbc4.CommunicationsException error is displayed in management server log after long time of inactive mysql connection
CLOUDSTACK-2654	VPC UI Missing information
CLOUDSTACK-2692	[MIPN][Enhancement] Add load balancing support for MIPN
CLOUDSTACK-2694	[Firewall Rule] Able to configure duplicate firewall rule with protocol and no ports
CLOUDSTACK-2702	unable to install XenServer Support Package (CSP) on Xen cloud platform 1.6
CLOUDSTACK-2721	Marvin cloudstackConnection fails if no port is specified
CLOUDSTACK-2722	KVM Hypervisor host install instructions say to set up VLANs for Basic networking
CLOUDSTACK-2780	primary storage is already mounted but not found in pool-list
CLOUDSTACK-2791	Installation instruction wrong
CLOUDSTACK-2795	Create template failed
CLOUDSTACK-2804	getEthByIp function in vpc_func.sh can return the wrong network interface
CLOUDSTACK-2845	Duplicate entries in the database since upgrade to 4.1.0
CLOUDSTACK-2860	Add new host into VMWare Cluster failed
CLOUDSTACK-2872	ubuntu 12.4 kvm issue CS 4.1 libvirt complaint and not able to start systemvm
CLOUDSTACK-2895	Can't start a VM with 3 volumes attached [VMWare]
CLOUDSTACK-2896	pt-BR docs without index.html
CLOUDSTACK-2897	Storage migration has left volumes on secondary storage
CLOUDSTACK-2907	In the case of a VMware HyperVisor,Sample Template (CentOS 5.3(64-bit) no GUI (vSphere)) download does not start.
CLOUDSTACK-2908	An automatic start setup is not carried out after cloudstack-usage installation.
CLOUDSTACK-2913	Spurious error message during install at the end of the RPM build process.
CLOUDSTACK-2926	An virtual machine instance cannot be started in a mixed hypervisor environment.
CLOUDSTACK-2929	unable to upgrade from version 3.0.6.20121222035904
CLOUDSTACK-2996	Nullpointer exception on view console on vmware
CLOUDSTACK-3008	Section 4.5.3.2 of the 4.1.0 documentation is incorrect cloud-client does not seem to exist
CLOUDSTACK-3048	Memory not reported correctly
CLOUDSTACK-3061	listHosts API doesn't has cpuused
CLOUDSTACK-3098	Location to copy vhd-util is wrong in docs
CLOUDSTACK-3138	Flaws in upgrade documentation from 3.0.2 -> 4.1.0
CLOUDSTACK-3143	revoke securitygroup ingress/egress does not return correct response
CLOUDSTACK-3163	KVM Virtual Router startup time is painfully long
CLOUDSTACK-3195	cannot view/delete forward rules if underlying/target VM is destroyed first
CLOUDSTACK-3216	Logs in the Software router are not being rotated
CLOUDSTACK-3243	Wrong NFS mount point in documentation
CLOUDSTACK-3272	EventBus: add global config parameters to specify which category of events are published on event bus.
CLOUDSTACK-3330	Autoscaling needs a VR to work
CLOUDSTACK-3350	Snapshots not deleted from swift
CLOUDSTACK-3366	Autoscaling won't scale down
CLOUDSTACK-3367	When one primary storage fails, all XenServer hosts get rebooted, killing all VMs, even those not on this primary storage.
CLOUDSTACK-3369	Autoscaling: Deleting an isolated network of an account deletes also the autoscaling rule for a network associated with another account
CLOUDSTACK-3381	Wrong instruction in CloudStack release notes
CLOUDSTACK-3383	GetHostStatsCommand fails when agent is running Ubuntu 13.04 (raring)
CLOUDSTACK-3401	Small page sizes return duplicate results
CLOUDSTACK-3421	When hypervisor is down, no HA occurs with log output "Agent state cannot be determined, do nothing"
CLOUDSTACK-3471	Provide an API to extract the log statements of a given jobid
CLOUDSTACK-3503	KVM Agent writes to agent.properties
CLOUDSTACK-3535	No HA actions are performed when a KVM host goes offline
CLOUDSTACK-3542	Ceph RBD provides wrong usage info to CloudStack
CLOUDSTACK-3543	the cpvm and ssvm' state is still running when the computer node was crashed
CLOUDSTACK-3546	[DOC] Update 4.2 Developer Guide
CLOUDSTACK-3556	Add NIC icon is not appearing in UI
CLOUDSTACK-3578	Unable to add Physical Dom0 as host, to cloudstack domU.
CLOUDSTACK-3593	Wrong Resource capacity for primary storage (Double the amount)
CLOUDSTACK-3625	Non CloudStack VM's = Alien VM's are deleted

Chapter 4. Upgrade Instructions

4.1. Upgrade from 4.1.0 to 4.1.1
4.2. Upgrade from 4.0.x to 4.1.1
4.3. Upgrade from 3.0.2 to 4.1.1
4.4. Upgrade from 2.2.14 to 4.1.1

This section contains upgrade instructions from prior versions of CloudStack to CloudStack 4.1.1. We include instructions on upgrading to CloudStack from pre-Apache versions of Citrix CloudStack (last version prior to Apache is 3.0.2) and from the releases made while CloudStack was in the Apache Incubator.

If you run into any issues during upgrades, please feel free to ask questions on users@cloudstack.apache.org or dev@cloudstack.apache.org.

4.1. Upgrade from 4.1.0 to 4.1.1

This section will guide you from CloudStack 4.1.0 versions to CloudStack 4.1.1.

Any steps that are hypervisor-specific will be called out with a note.

We recommend reading through this section once or twice before beginning your upgrade procedure, and working through it on a test system before working on a production system.

Most users of CloudStack manage the installation and upgrades of CloudStack with one of Linux's predominant package systems, RPM or APT. This guide assumes you'll be using RPM and Yum (for Red Hat Enterprise Linux or CentOS), or APT and Debian packages (for Ubuntu).
Create RPM or Debian packages (as appropriate) and a repository from the 4.1.1 source, or check the CloudStack downloads page at http://cloudstack.apache.org/downloads.html for package repositories supplied by community members. You will need them for step 5 or step 6.
Instructions for creating packages from the CloudStack source are in the Installation Guide.
Stop your management server or servers. Run this on all management server hosts:
```
# service cloudstack-management stop
```
If you are running a usage server or usage servers, stop those as well:
```
# service cloudstack-usage stop
```
Make a backup of your MySQL database. If you run into any issues or need to roll back the upgrade, this will assist in debugging or restoring your existing environment. You'll be prompted for your password.
```
# mysqldump -u root -p cloud > cloudstack-backup.sql
```
If you are using Ubuntu, follow this procedure to upgrade your packages. If not, skip to step 6.
Community Packages
This section assumes you're using the community supplied packages for CloudStack. If you've created your own packages and APT repository, substitute your own URL for the ones used in these examples.
1. The first step is to confirm that your sources list is pointed to the correct repository for each system with CloudStack packages. This means all management servers, and any hosts that have the KVM agent. (No changes should be necessary for hosts that are running VMware or Xen.)
  Start by opening /etc/apt/sources.list.d/cloudstack.list on any systems that have CloudStack packages installed.
  This file should have one line, which contains:
```
deb http://cloudstack.apt-get.eu/ubuntu precise 4.1
```
  If the line above is not correct, correct it. Note that if you're using your own package repository, change this line to read as appropriate for your 4.1.1 repository.
2. Now update your apt package list:
```
$ sudo apt-get update
```
3. Now that you have the repository configured, it's time to update the cloudstack-management package. This will pull in any other dependencies and updates you need.
```
$ sudo apt-get install cloudstack-management
```
4. On KVM hosts you will need to also upgrade the cloudstack-agent package:
```
$ sudo apt-get install cloudstack-agent
```
5. Verify that the file /etc/cloudstack/agent/environment.properties has a line that reads:
```
paths.script=/usr/share/cloudstack-common
```
  If not, add the line.
6. Restart the agent:
```
                                service cloudstack-agent stop
                                killall jsvc
                                service cloudstack-agent start
```
If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If not, skip to step 7.
Community Packages
This section assumes you're using the community supplied packages for CloudStack. If you've created your own packages and yum repository, substitute your own URL for the ones used in these examples.
1. The first step is to confirm that your yum configuration is pointed at the correct repository for each system with CloudStack packages. This means all management servers, and any hosts that have the KVM agent. (No changes should be necessary for hosts that are running VMware or Xen.)
  Start by opening /etc/yum.repos.d/cloudstack.repo on any systems that have CloudStack packages installed.
  This file should have content similar to the following:
```
                                [apache-cloudstack]
                                name=Apache CloudStack
                                baseurl=http://cloudstack.apt-get.eu/rhel/4.1/
                                enabled=1
                                gpgcheck=0
```
  If the file above is not correct, correct it. Note that if you're using your own package repository, change the baseurl line to read as appropriate for your 4.1.1 repository.
2. Now that you have the repository configured, it's time to update the cloudstack-management package.
```
$ sudo yum update cloudstack-management
```
3. For KVM hosts, you will need to update the cloudstack-agent package.
```
$ sudo yum update cloudstack-agent
```
4. Verify that the file /etc/cloudstack/agent/environment.properties has a line that reads:
```
paths.script=/usr/share/cloudstack-common
```
  If not, add the line.
5. Restart the agent:
```
                                service cloudstack-agent stop
                                killall jsvc
                                service cloudstack-agent start
```

Once you've upgraded the packages on your management servers, you'll need to restart the system VMs. Make sure port 8096 is open in your local host firewall to do this.

There is a script that will do this for you, all you need to do is run the script and supply the IP address for your MySQL instance and your MySQL credentials:

# nohup cloudstack-sysvmadm -d IP address -u cloud -p -a > sysvm.log 2>&1 &

You can monitor the log for progress. The process of restarting the system VMs can take an hour or more.

# tail -f sysvm.log

The output to sysvm.log will look something like this:

                        Stopping and starting 1 secondary storage vm(s)...
                        Done stopping and starting secondary storage vm(s)
                        Stopping and starting 1 console proxy vm(s)...
                        Done stopping and starting console proxy vm(s).
                        Stopping and starting 4 running routing vm(s)...
                        Done restarting router(s).

4.2. Upgrade from 4.0.x to 4.1.1

This section will guide you from CloudStack 4.0.x versions to CloudStack 4.1.1.

Any steps that are hypervisor-specific will be called out with a note.

Package Structure Changes

The package structure for CloudStack has changed significantly since the 4.0.x releases. If you've compiled your own packages, you'll notice that the package names and the number of packages has changed. This is not a bug.

However, this does mean that the procedure is not as simple as an apt-get upgrade or yum update, so please follow this section carefully.

We recommend reading through this section once or twice before beginning your upgrade procedure, and working through it on a test system before working on a production system.

Most users of CloudStack manage the installation and upgrades of CloudStack with one of Linux's predominant package systems, RPM or APT. This guide assumes you'll be using RPM and Yum (for Red Hat Enterprise Linux or CentOS), or APT and Debian packages (for Ubuntu).
Create RPM or Debian packages (as appropriate) and a repository from the 4.1.1 source, or check the CloudStack downloads page at http://cloudstack.apache.org/downloads.html for package repositories supplied by community members. You will need them for step 7 or step 8.
Instructions for creating packages from the CloudStack source are in the Installation Guide.
Stop your management server or servers. Run this on all management server hosts:
```
# service cloud-management stop
```
If you are running a usage server or usage servers, stop those as well:
```
# service cloud-usage stop
```
Make a backup of your MySQL database. If you run into any issues or need to roll back the upgrade, this will assist in debugging or restoring your existing environment. You'll be prompted for your password.
```
# mysqldump -u root -p cloud > cloudstack-backup.sql
```
If you have made changes to /etc/cloud/management/components.xml, you'll need to carry these over manually to the new file, /etc/cloudstack/management/componentContext.xml. This is not done automatically. (If you're unsure, we recommend making a backup of the original components.xml to be on the safe side.
After upgrading to 4.1, API clients are expected to send plain text passwords for login and user creation, instead of MD5 hash. Incase, api client changes are not acceptable, following changes are to be made for backward compatibility:
Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default authenticator (1st entry in the userAuthenticators adapter list is default)
```

<bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList">
  <property name="Adapters">
    <list>
      <ref bean="PlainTextUserAuthenticator"/>
      <ref bean="MD5UserAuthenticator"/>
      <ref bean="LDAPUserAuthenticator"/>
    </list>
  </property>
</bean>
```
PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to 4.1.
If you are using Ubuntu, follow this procedure to upgrade your packages. If not, skip to step 8.
Community Packages
This section assumes you're using the community supplied packages for CloudStack. If you've created your own packages and APT repository, substitute your own URL for the ones used in these examples.
1. The first order of business will be to change the sources list for each system with CloudStack packages. This means all management servers, and any hosts that have the KVM agent. (No changes should be necessary for hosts that are running VMware or Xen.)
  Start by opening /etc/apt/sources.list.d/cloudstack.list on any systems that have CloudStack packages installed.
  This file should have one line, which contains:
```
deb http://cloudstack.apt-get.eu/ubuntu precise 4.0
```
  We'll change it to point to the new package repository:
```
deb http://cloudstack.apt-get.eu/ubuntu precise 4.1
```
  If you're using your own package repository, change this line to read as appropriate for your 4.1.1 repository.
2. Now update your apt package list:
```
$ sudo apt-get update
```
3. Now that you have the repository configured, it's time to install the cloudstack-management package. This will pull in any other dependencies you need.
```
$ sudo apt-get install cloudstack-management
```
4. On KVM hosts you will need to manually install the cloudstack-agent package:
```
$ sudo apt-get install cloudstack-agent
```
  During the installation of cloudstack-agent, APT will copy your agent.properties, log4j-cloud.xml, and environment.properties from /etc/cloud/agent to /etc/cloudstack/agent.
  When prompted whether you wish to keep your configuration, say Yes.
5. Verify that the file /etc/cloudstack/agent/environment.properties has a line that reads:
```
paths.script=/usr/share/cloudstack-common
```
  If not, add the line.
6. Restart the agent:
```
                                service cloud-agent stop
                                killall jsvc
                                service cloudstack-agent start
```
7. During the upgrade, log4j-cloud.xml was simply copied over, so the logs will continue to be added to /var/log/cloud/agent/agent.log. There's nothing wrong with this, but if you prefer to be consistent, you can change this by copying over the sample configuration file:
```
                                cd /etc/cloudstack/agent
                                mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
                                service cloudstack-agent restart
```
8. Once the agent is running, you can uninstall the old cloud-* packages from your system:
```
sudo dpkg --purge cloud-agent
```
If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If not, skip to step 9.
Community Packages
This section assumes you're using the community supplied packages for CloudStack. If you've created your own packages and yum repository, substitute your own URL for the ones used in these examples.
1. The first order of business will be to change the yum repository for each system with CloudStack packages. This means all management servers, and any hosts that have the KVM agent. (No changes should be necessary for hosts that are running VMware or Xen.)
  Start by opening /etc/yum.repos.d/cloudstack.repo on any systems that have CloudStack packages installed.
  This file should have content similar to the following:
```
                                [apache-cloudstack]
                                name=Apache CloudStack
                                baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
                                enabled=1
                                gpgcheck=0
```
  If you are using the community provided package repository, change the baseurl to http://cloudstack.apt-get.eu/rhel/4.1/
  If you're using your own package repository, change this line to read as appropriate for your 4.1.1 repository.
2. Now that you have the repository configured, it's time to install the cloudstack-management package by upgrading the older cloud-client package.
```
$ sudo yum upgrade cloud-client
```
3. For KVM hosts, you will need to upgrade the cloud-agent package, similarly installing the new version as cloudstack-agent.
```
$ sudo yum upgrade cloud-agent
```
  During the installation of cloudstack-agent, the RPM will copy your agent.properties, log4j-cloud.xml, and environment.properties from /etc/cloud/agent to /etc/cloudstack/agent.
4. Verify that the file /etc/cloudstack/agent/environment.properties has a line that reads:
```
paths.script=/usr/share/cloudstack-common
```
  If not, add the line.
5. Restart the agent:
```
                                service cloud-agent stop
                                killall jsvc
                                service cloudstack-agent start
```

Once you've upgraded the packages on your management servers, you'll need to restart the system VMs. Make sure port 8096 is open in your local host firewall to do this.

There is a script that will do this for you, all you need to do is run the script and supply the IP address for your MySQL instance and your MySQL credentials:

# nohup cloudstack-sysvmadm -d IP address -u cloud -p -a > sysvm.log 2>&1 &

You can monitor the log for progress. The process of restarting the system VMs can take an hour or more.

# tail -f sysvm.log

The output to sysvm.log will look something like this:

                        Stopping and starting 1 secondary storage vm(s)...
                        Done stopping and starting secondary storage vm(s)
                        Stopping and starting 1 console proxy vm(s)...
                        Done stopping and starting console proxy vm(s).
                        Stopping and starting 4 running routing vm(s)...
                        Done restarting router(s).

For Xen Hosts: Copy vhd-utils
This step is only for CloudStack installs that are using Xen hosts.
Copy the file vhd-utils to /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver.

4.3. Upgrade from 3.0.2 to 4.1.1

This section will guide you from Citrix CloudStack 3.0.2 to CloudStack 4.1.1. Sections that are hypervisor-specific will be called out with a note.

Note

The following upgrade instructions apply only if you're using VMware hosts. If you're not using VMware hosts, skip this step and move on to 2.

In each zone that includes VMware hosts, you need to add a new system VM template.

While running the existing 3.0.2 system, log in to the UI as root administrator.
In the left navigation bar, click Templates.
In Select view, click Templates.
Click Register template.
The Register template dialog box is displayed.

In the Register template dialog box, specify the following values (do not change these):

Field	Value
Name	systemvm-vmware-4.0
Description	systemvm-vmware-4.0
URL	http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova
Zone	Choose the zone where this hypervisor is used
Hypervisor	VMware
Format	OVA
OS Type	Debian GNU/Linux 5.0 (32-bit)
Extractable	no
Password Enabled	no
Public	no
Featured	no

Watch the screen to be sure that the template downloads successfully and enters the READY state. Do not proceed until this is successful.

Stop all Usage Servers if running. Run this on all Usage Server hosts.
```
# service cloud-usage stop
```
Stop the Management Servers. Run this on all Management Server hosts.
```
# service cloud-management stop
```
On the MySQL master, take a backup of the MySQL databases. We recommend performing this step even in test upgrades. If there is an issue, this will assist with debugging.
In the following commands, it is assumed that you have set the root password on the database, which is a CloudStack recommended best practice. Substitute your own MySQL root password.
```
# mysqldump -u root -pmysql_password cloud > cloud-backup.dmp
                        # mysqldump -u root -pmysql_password cloud_usage > cloud-usage-backup.dmp
```
Either build RPM/DEB packages as detailed in the Installation Guide, or use one of the community provided yum/apt repositories to gain access to the CloudStack binaries.
If you are using Ubuntu, follow this procedure to upgrade your packages. If not, skip to step 7.
Community Packages
This section assumes you're using the community supplied packages for CloudStack. If you've created your own packages and APT repository, substitute your own URL for the ones used in these examples.
1. The first order of business will be to change the sources list for each system with CloudStack packages. This means all management servers, and any hosts that have the KVM agent. (No changes should be necessary for hosts that are running VMware or Xen.)
  Start by opening /etc/apt/sources.list.d/cloudstack.list on any systems that have CloudStack packages installed.
  This file should have one line, which contains:
```
deb http://cloudstack.apt-get.eu/ubuntu precise 4.0
```
  We'll change it to point to the new package repository:
```
deb http://cloudstack.apt-get.eu/ubuntu precise 4.1
```
  If you're using your own package repository, change this line to read as appropriate for your 4.1.1 repository.
2. Now update your apt package list:
```
$ sudo apt-get update
```
3. Now that you have the repository configured, it's time to install the cloudstack-management package. This will pull in any other dependencies you need.
```
$ sudo apt-get install cloudstack-management
```
4. On KVM hosts you will need to manually install the cloudstack-agent package:
```
$ sudo apt-get install cloudstack-agent
```
  During the installation of cloudstack-agent, APT will copy your agent.properties, log4j-cloud.xml, and environment.properties from /etc/cloud/agent to /etc/cloudstack/agent.
  When prompted whether you wish to keep your configuration, say Yes.
5. Verify that the file /etc/cloudstack/agent/environment.properties has a line that reads:
```
paths.script=/usr/share/cloudstack-common
```
  If not, add the line.
6. Restart the agent:
```
                                service cloud-agent stop
                                killall jsvc
                                service cloudstack-agent start
```
7. During the upgrade, log4j-cloud.xml was simply copied over, so the logs will continue to be added to /var/log/cloud/agent/agent.log. There's nothing wrong with this, but if you prefer to be consistent, you can change this by copying over the sample configuration file:
```
                                cd /etc/cloudstack/agent
                                mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
                                service cloudstack-agent restart
```
8. Once the agent is running, you can uninstall the old cloud-* packages from your system:
```
sudo dpkg --purge cloud-agent
```
If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If not, skip to step 8.
Community Packages
This section assumes you're using the community supplied packages for CloudStack. If you've created your own packages and yum repository, substitute your own URL for the ones used in these examples.
1. The first order of business will be to change the yum repository for each system with CloudStack packages. This means all management servers, and any hosts that have the KVM agent. (No changes should be necessary for hosts that are running VMware or Xen.)
  Start by opening /etc/yum.repos.d/cloudstack.repo on any systems that have CloudStack packages installed.
  This file should have content similar to the following:
```
                                [apache-cloudstack]
                                name=Apache CloudStack
                                baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
                                enabled=1
                                gpgcheck=0
```
  If you are using the community provided package repository, change the baseurl to http://cloudstack.apt-get.eu/rhel/4.1/
  If you're using your own package repository, change this line to read as appropriate for your 4.1.1 repository.
2. Now that you have the repository configured, it's time to install the cloudstack-management package by upgrading the older cloud-client package.
```
$ sudo yum upgrade cloud-client
```
3. For KVM hosts, you will need to upgrade the cloud-agent package, similarly installing the new version as cloudstack-agent.
```
$ sudo yum upgrade cloud-agent
```
  During the installation of cloudstack-agent, the RPM will copy your agent.properties, log4j-cloud.xml, and environment.properties from /etc/cloud/agent to /etc/cloudstack/agent.
4. Verify that the file /etc/cloudstack/agent/environment.properties has a line that reads:
```
paths.script=/usr/share/cloudstack-common
```
  If not, add the line.
5. Restart the agent:
```
                                service cloud-agent stop
                                killall jsvc
                                service cloudstack-agent start
```
If you have made changes to your copy of /etc/cloud/management/components.xml the changes will be preserved in the upgrade. However, you need to do the following steps to place these changes in a new version of the file which is compatible with version 4.1.1.
1. Make a backup copy of /etc/cloud/management/components.xml. For example:
```
# mv /etc/cloud/management/components.xml /etc/cloud/management/components.xml-backup
```
2. Copy /etc/cloud/management/components.xml.rpmnew to create a new /etc/cloud/management/components.xml:
```
# cp -ap /etc/cloud/management/components.xml.rpmnew /etc/cloud/management/components.xml
```
3. Merge your changes from the backup file into the new components.xml.
```
# vi /etc/cloud/management/components.xml
```
Note
If you have more than one management server node, repeat the upgrade steps on each node.
After upgrading to 4.1, API clients are expected to send plain text passwords for login and user creation, instead of MD5 hash. Incase, api client changes are not acceptable, following changes are to be made for backward compatibility:
Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default authenticator (1st entry in the userAuthenticators adapter list is default)
```

<bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList">
  <property name="Adapters">
    <list>
      <ref bean="PlainTextUserAuthenticator"/>
      <ref bean="MD5UserAuthenticator"/>
      <ref bean="LDAPUserAuthenticator"/>
    </list>
  </property>
</bean>
```
PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to 4.1.
Start the first Management Server. Do not start any other Management Server nodes yet.
```
# service cloudstack-management start
```
Wait until the databases are upgraded. Ensure that the database upgrade is complete. After confirmation, start the other Management Servers one at a time by running the same command on each node.
Note
Failing to restart the Management Server indicates a problem in the upgrade. Having the Management Server restarted without any issues indicates that the upgrade is successfully completed.
Start all Usage Servers (if they were running on your previous version). Perform this on each Usage Server host.
# service cloudstack-usage start
Note
Additional steps are required for each KVM host. These steps will not affect running guests in the cloud. These steps are required only for clouds using KVM as hosts and only on the KVM hosts.
1. Configure a yum or apt respository containing the CloudStack packages as outlined in the Installation Guide.
2. Stop the running agent.
  # service cloud-agent stop
3. Update the agent software with one of the following command sets as appropriate for your environment.
  # yum update cloud*
  # apt-get update
  # apt-get upgrade cloud*
4. Start the agent.
```
# service cloudstack-agent start
```
5. Edit /etc/cloud/agent/agent.properties to change the resource parameter from "com.cloud.agent.resource.computing.LibvirtComputingResource" to "com.cloud.hypervisor.kvm.resource.LibvirtComputingResource".
6. Start the cloud agent and cloud management services.
7. When the Management Server is up and running, log in to the CloudStack UI and restart the virtual router for proper functioning of all the features.
Log in to the CloudStack UI as administrator, and check the status of the hosts. All hosts should come to Up state (except those that you know to be offline). You may need to wait 20 or 30 minutes, depending on the number of hosts.
Note
Troubleshooting: If login fails, clear your browser cache and reload the page.

Do not proceed to the next step until the hosts show in Up state.
If you are upgrading from 3.0.2, perform the following:
1. Ensure that the admin port is set to 8096 by using the "integration.api.port" global parameter.
  This port is used by the cloud-sysvmadm script at the end of the upgrade procedure. For information about how to set this parameter, see "Setting Global Configuration Parameters" in the Installation Guide.
2. Restart the Management Server.
  Note
  If you don't want the admin port to remain open, you can set it to null after the upgrade is done and restart the management server.
Run the cloud-sysvmadm script to stop, then start, all Secondary Storage VMs, Console Proxy VMs, and virtual routers. Run the script once on each management server. Substitute your own IP address of the MySQL instance, the MySQL user to connect as, and the password to use for that user. In addition to those parameters, provide the -c and -r arguments. For example:
# nohup cloud-sysvmadm -d 192.168.1.5 -u cloud -p password -c -r > sysvm.log 2>&1 &
# tail -f sysvm.log
This might take up to an hour or more to run, depending on the number of accounts in the system.
If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version supported by CloudStack 4.1.1. The supported versions are XenServer 5.6 SP2 and 6.0.2. Instructions for upgrade can be found in the CloudStack 4.1.1 Installation Guide under "Upgrading XenServer Versions."

Now apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to XenServer v6.0.2 hypervisor hosts.

Disconnect the XenServer cluster from CloudStack.
In the left navigation bar of the CloudStack UI, select Infrastructure. Under Clusters, click View All. Select the XenServer cluster and click Actions - Unmanage.
This may fail if there are hosts not in one of the states Up, Down, Disconnected, or Alert. You may need to fix that before unmanaging this cluster.
Wait until the status of the cluster has reached Unmanaged. Use the CloudStack UI to check on the status. When the cluster is in the unmanaged state, there is no connection to the hosts in the cluster.
To clean up the VLAN, log in to one XenServer host and run:
/opt/xensource/bin/cloud-clean-vlan.sh
Now prepare the upgrade by running the following on one XenServer host:
/opt/xensource/bin/cloud-prepare-upgrade.sh
If you see a message like "can't eject CD", log in to the VM and unmount the CD, then run this script again.
Upload the hotfix to the XenServer hosts. Always start with the Xen pool master, then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the hotfixes to the host. Place them in a temporary folder such as /tmp.
On the Xen pool master, upload the hotfix with this command:
xe patch-upload file-name=XS602E003.xsupdate
Make a note of the output from this command, which is a UUID for the hotfix file. You'll need it in another step later.
Note
(Optional) If you are applying other hotfixes as well, you can repeat the commands in this section with the appropriate hotfix number. For example, XS602E004.xsupdate.
Manually live migrate all VMs on this host to another host. First, get a list of the VMs on this host:
# xe vm-list
Then use this command to migrate each VM. Replace the example host name and VM name with your own:
# xe vm-migrate live=true host=host-name vm=VM-name
Troubleshooting
If you see a message like "You attempted an operation on a VM which requires PV drivers to be installed but the drivers were not detected," run:
/opt/xensource/bin/make_migratable.sh b6cf79c8-02ee-050b-922f-49583d9f1a14.
Apply the hotfix. First, get the UUID of this host:
```
# xe host-list
```
Then use the following command to apply the hotfix. Replace the example host UUID with the current host ID, and replace the hotfix UUID with the output from the patch-upload command you ran on this machine earlier. You can also get the hotfix UUID by running xe patch-list.
```
xe patch-apply host-uuid=host-uuid uuid=hotfix-uuid
```

Copy the following files from the CloudStack Management Server to the host.

Copy from here...	...to here
/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py	/opt/xensource/sm/NFSSR.py
/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/setupxenserver.sh	/opt/xensource/bin/setupxenserver.sh
/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/make_migratable.sh	/opt/xensource/bin/make_migratable.sh

(Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud Support Pack.
- Download the CSP software onto the XenServer host from one of the following links:
  For hotfix XS602E005: http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz
  For hotfix XS602E007: http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz
- Extract the file:
```
# tar xf xenserver-cloud-supp.tgz
```
- Run the following script:
```
# xe-install-supplemental-pack xenserver-cloud-supp.iso
```
- If the XenServer host is part of a zone that uses basic networking, disable Open vSwitch (OVS):
```
# xe-switch-network-backend  bridge
```
Reboot this XenServer host.
Run the following:
```
/opt/xensource/bin/setupxenserver.sh
```
Note
If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or directory" appears, you can safely ignore it.

Run the following:

for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk '{print $NF}'`; do xe pbd-plug uuid=$pbd ;

On each slave host in the Xen pool, repeat these steps, starting from "manually live migrate VMs."

Troubleshooting Tip

If passwords which you know to be valid appear not to work after upgrade, or other UI issues are seen, try clearing your browser cache and reloading the UI page.

4.4. Upgrade from 2.2.14 to 4.1.1

Ensure that you query your IPaddress usage records and process them; for example, issue invoices for any usage that you have not yet billed users for.
Starting in 3.0.2, the usage record format for IP addresses is the same as the rest of the usage types. Instead of a single record with the assignment and release dates, separate records are generated per aggregation period with start and end dates. After upgrading to 4.1.1, any existing IP address usage records in the old format will no longer be available.
If you are using version 2.2.0 - 2.2.13, first upgrade to 2.2.14 by using the instructions in the 2.2.14 Release Notes.
KVM Hosts
If KVM hypervisor is used in your cloud, be sure you completed the step to insert a valid username and password into the host_details table on each KVM node as described in the 2.2.14 Release Notes. This step is critical, as the database will be encrypted after the upgrade to 4.1.1.
While running the 2.2.14 system, log in to the UI as root administrator.

Using the UI, add a new System VM template for each hypervisor type that is used in your cloud. In each zone, add a system VM template for each hypervisor used in that zone

In the left navigation bar, click Templates.
In Select view, click Templates.
Click Register template.
The Register template dialog box is displayed.

In the Register template dialog box, specify the following values depending on the hypervisor type (do not change these):

Hypervisor	Description
XenServer	Name: systemvm-xenserver-3.0.0 Description: systemvm-xenserver-3.0.0 URL: http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2 Zone: Choose the zone where this hypervisor is used Hypervisor: XenServer Format: VHD OS Type: Debian GNU/Linux 5.0 (32-bit) Extractable: no Password Enabled: no Public: no Featured: no
KVM	Name: systemvm-kvm-3.0.0 Description: systemvm-kvm-3.0.0 URL: http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2 Zone: Choose the zone where this hypervisor is used Hypervisor: KVM Format: QCOW2 OS Type: Debian GNU/Linux 5.0 (32-bit) Extractable: no Password Enabled: no Public: no Featured: no
VMware	Name: systemvm-vmware-4.0 Description: systemvm-vmware-4.0 URL: http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova Zone: Choose the zone where this hypervisor is used Hypervisor: VMware Format: OVA OS Type: Debian GNU/Linux 5.0 (32-bit) Extractable: no Password Enabled: no Public: no Featured: no

Watch the screen to be sure that the template downloads successfully and enters the READY state. Do not proceed until this is successful
WARNING: If you use more than one type of hypervisor in your cloud, be sure you have repeated these steps to download the system VM template for each hypervisor type. Otherwise, the upgrade will fail.
Stop all Usage Servers if running. Run this on all Usage Server hosts.
```
# service cloud-usage stop
```
Stop the Management Servers. Run this on all Management Server hosts.
```
# service cloud-management stop
```
On the MySQL master, take a backup of the MySQL databases. We recommend performing this step even in test upgrades. If there is an issue, this will assist with debugging.
In the following commands, it is assumed that you have set the root password on the database, which is a CloudStack recommended best practice. Substitute your own MySQL root password.
```
# mysqldump -u root -pmysql_password cloud > cloud-backup.dmp
                        # mysqldump -u root -pmysql_password cloud_usage > cloud-usage-backup.dmp
```
Either build RPM/DEB packages as detailed in the Installation Guide, or use one of the community provided yum/apt repositories to gain access to the CloudStack binaries.
If you are using Ubuntu, follow this procedure to upgrade your packages. If not, skip to step 12.
Community Packages
This section assumes you're using the community supplied packages for CloudStack. If you've created your own packages and APT repository, substitute your own URL for the ones used in these examples.
1. The first order of business will be to change the sources list for each system with CloudStack packages. This means all management servers, and any hosts that have the KVM agent. (No changes should be necessary for hosts that are running VMware or Xen.)
  Start by opening /etc/apt/sources.list.d/cloudstack.list on any systems that have CloudStack packages installed.
  This file should have one line, which contains:
```
deb http://cloudstack.apt-get.eu/ubuntu precise 4.0
```
  We'll change it to point to the new package repository:
```
deb http://cloudstack.apt-get.eu/ubuntu precise 4.1
```
  If you're using your own package repository, change this line to read as appropriate for your 4.1.1 repository.
2. Now update your apt package list:
```
$ sudo apt-get update
```
3. Now that you have the repository configured, it's time to install the cloudstack-management package. This will pull in any other dependencies you need.
```
$ sudo apt-get install cloudstack-management
```
4. on KVM hosts you will need to manually install the cloudstack-agent package:
```
$ sudo apt-get install cloudstack-agent
```
  During the installation of cloudstack-agent, APT will copy your agent.properties, log4j-cloud.xml, and environment.properties from /etc/cloud/agent to /etc/cloudstack/agent.
  When prompted whether you wish to keep your configuration, say Yes.
5. Verify that the file /etc/cloudstack/agent/environment.properties has a line that reads:
```
paths.script=/usr/share/cloudstack-common
```
  If not, add the line.
6. Restart the agent:
```
                                service cloud-agent stop
                                killall jsvc
                                service cloudstack-agent start
```
7. During the upgrade, log4j-cloud.xml was simply copied over, so the logs will continue to be added to /var/log/cloud/agent/agent.log. There's nothing wrong with this, but if you prefer to be consistent, you can change this by copying over the sample configuration file:
```
                                cd /etc/cloudstack/agent
                                mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
                                service cloudstack-agent restart
```
8. Once the agent is running, you can uninstall the old cloud-* packages from your system:
```
sudo dpkg --purge cloud-agent
```
If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If not, skip to step 13.
Community Packages
This section assumes you're using the community supplied packages for CloudStack. If you've created your own packages and yum repository, substitute your own URL for the ones used in these examples.
1. The first order of business will be to change the yum repository for each system with CloudStack packages. This means all management servers, and any hosts that have the KVM agent. (No changes should be necessary for hosts that are running VMware or Xen.)
  Start by opening /etc/yum.repos.d/cloudstack.repo on any systems that have CloudStack packages installed.
  This file should have content similar to the following:
```
                                [apache-cloudstack]
                                name=Apache CloudStack
                                baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
                                enabled=1
                                gpgcheck=0
```
  If you are using the community provided package repository, change the baseurl to http://cloudstack.apt-get.eu/rhel/4.1/
  If you're using your own package repository, change this line to read as appropriate for your 4.1.1 repository.
2. Now that you have the repository configured, it's time to install the cloudstack-management package by upgrading the older cloud-client package.
```
$ sudo yum upgrade cloud-client
```
3. For KVM hosts, you will need to upgrade the cloud-agent package, similarly installing the new version as cloudstack-agent.
```
$ sudo yum upgrade cloud-agent
```
  During the installation of cloudstack-agent, the RPM will copy your agent.properties, log4j-cloud.xml, and environment.properties from /etc/cloud/agent to /etc/cloudstack/agent.
4. Verify that the file /etc/cloudstack/agent/environment.properties has a line that reads:
```
paths.script=/usr/share/cloudstack-common
```
  If not, add the line.
5. Restart the agent:
```
                                service cloud-agent stop
                                killall jsvc
                                service cloudstack-agent start
```
If you have made changes to your existing copy of the file components.xml in your previous-version CloudStack installation, the changes will be preserved in the upgrade. However, you need to do the following steps to place these changes in a new version of the file which is compatible with version 4.0.0-incubating.
Note
How will you know whether you need to do this? If the upgrade output in the previous step included a message like the following, then some custom content was found in your old components.xml, and you need to merge the two files:
```
warning: /etc/cloud/management/components.xml created as /etc/cloud/management/components.xml.rpmnew
```
1. Make a backup copy of your /etc/cloud/management/components.xml file. For example:
```
# mv /etc/cloud/management/components.xml /etc/cloud/management/components.xml-backup
```
2. Copy /etc/cloud/management/components.xml.rpmnew to create a new /etc/cloud/management/components.xml:
```
# cp -ap /etc/cloud/management/components.xml.rpmnew /etc/cloud/management/components.xml
```
3. Merge your changes from the backup file into the new components.xml file.
```
# vi /etc/cloud/management/components.xml
```
After upgrading to 4.1, API clients are expected to send plain text passwords for login and user creation, instead of MD5 hash. Incase, api client changes are not acceptable, following changes are to be made for backward compatibility:
Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default authenticator (1st entry in the userAuthenticators adapter list is default)
```

<bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList">
  <property name="Adapters">
    <list>
      <ref bean="PlainTextUserAuthenticator"/>
      <ref bean="MD5UserAuthenticator"/>
      <ref bean="LDAPUserAuthenticator"/>
    </list>
  </property>
</bean>
```
PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to 4.1.
If you have made changes to your existing copy of the /etc/cloud/management/db.properties file in your previous-version CloudStack installation, the changes will be preserved in the upgrade. However, you need to do the following steps to place these changes in a new version of the file which is compatible with version 4.0.0-incubating.
1. Make a backup copy of your file /etc/cloud/management/db.properties. For example:
```
# mv /etc/cloud/management/db.properties /etc/cloud/management/db.properties-backup
```
2. Copy /etc/cloud/management/db.properties.rpmnew to create a new /etc/cloud/management/db.properties:
```
# cp -ap /etc/cloud/management/db.properties.rpmnew etc/cloud/management/db.properties
```
3. Merge your changes from the backup file into the new db.properties file.
```
# vi /etc/cloud/management/db.properties
```
On the management server node, run the following command. It is recommended that you use the command-line flags to provide your own encryption keys. See Password and Key Encryption in the Installation Guide.
```
# cloud-setup-encryption -e encryption_type -m management_server_key -k database_key
```
When used without arguments, as in the following example, the default encryption type and keys will be used:
- (Optional) For encryption_type, use file or web to indicate the technique used to pass in the database encryption password. Default: file.
- (Optional) For management_server_key, substitute the default key that is used to encrypt confidential parameters in the properties file. Default: password. It is highly recommended that you replace this with a more secure value
- (Optional) For database_key, substitute the default key that is used to encrypt confidential parameters in the CloudStack database. Default: password. It is highly recommended that you replace this with a more secure value.
Repeat steps 10 - 14 on every management server node. If you provided your own encryption key in step 14, use the same key on all other management servers.
Start the first Management Server. Do not start any other Management Server nodes yet.
```
# service cloudstack-management start
```
Wait until the databases are upgraded. Ensure that the database upgrade is complete. You should see a message like "Complete! Done." After confirmation, start the other Management Servers one at a time by running the same command on each node.
Start all Usage Servers (if they were running on your previous version). Perform this on each Usage Server host.
```
# service cloudstack-usage start
```
(KVM only) Additional steps are required for each KVM host. These steps will not affect running guests in the cloud. These steps are required only for clouds using KVM as hosts and only on the KVM hosts.
1. Configure your CloudStack package repositories as outlined in the Installation Guide
2. Stop the running agent.
```
# service cloud-agent stop
```
3. Update the agent software with one of the following command sets as appropriate.
```
# yum update cloud*
```
```
                                # apt-get update
                                # apt-get upgrade cloud*
```
4. Start the agent.
```
# service cloudstack-agent start
```
5. Copy the contents of the agent.properties file to the new agent.properties file by using the following command
```
sed -i 's/com.cloud.agent.resource.computing.LibvirtComputingResource/com.cloud.hypervisor.kvm.resource.LibvirtComputingResource/g' /etc/cloud/agent/agent.properties
```
6. Start the cloud agent and cloud management services.
7. When the Management Server is up and running, log in to the CloudStack UI and restart the virtual router for proper functioning of all the features.
Log in to the CloudStack UI as admin, and check the status of the hosts. All hosts should come to Up state (except those that you know to be offline). You may need to wait 20 or 30 minutes, depending on the number of hosts.
Do not proceed to the next step until the hosts show in the Up state. If the hosts do not come to the Up state, contact support.

Run the following script to stop, then start, all Secondary Storage VMs, Console Proxy VMs, and virtual routers.

Run the command once on one management server. Substitute your own IP address of the MySQL instance, the MySQL user to connect as, and the password to use for that user. In addition to those parameters, provide the "-c" and "-r" arguments. For example:
```
# nohup cloud-sysvmadm -d 192.168.1.5 -u cloud -p password -c -r > sysvm.log 2>&1 &
                                # tail -f sysvm.log
```
This might take up to an hour or more to run, depending on the number of accounts in the system.

After the script terminates, check the log to verify correct execution:

# tail -f sysvm.log

The content should be like the following:

                                Stopping and starting 1 secondary storage vm(s)...
                                Done stopping and starting secondary storage vm(s)
                                Stopping and starting 1 console proxy vm(s)...
                                Done stopping and starting console proxy vm(s).
                                Stopping and starting 4 running routing vm(s)...
                                Done restarting router(s).

If you would like additional confirmation that the new system VM templates were correctly applied when these system VMs were rebooted, SSH into the System VM and check the version.
Use one of the following techniques, depending on the hypervisor.
XenServer or KVM:
SSH in by using the link local IP address of the system VM. For example, in the command below, substitute your own path to the private key used to log in to the system VM and your own link local IP.
Run the following commands on the XenServer or KVM host on which the system VM is present:
```
# ssh -i private-key-path link-local-ip -p 3922
                        # cat /etc/cloudstack-release
```
The output should be like the following:
```
Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012
```
ESXi
SSH in using the private IP address of the system VM. For example, in the command below, substitute your own path to the private key used to log in to the system VM and your own private IP.
Run the following commands on the Management Server:
```
# ssh -i private-key-path private-ip -p 3922
                        # cat /etc/cloudstack-release
```
The output should be like the following:
```
Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012
```
If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version supported by CloudStack 4.0.0-incubating. The supported versions are XenServer 5.6 SP2 and 6.0.2. Instructions for upgrade can be found in the CloudStack 4.0.0-incubating Installation Guide.

Apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to XenServer v6.0.2 hypervisor hosts.

Disconnect the XenServer cluster from CloudStack.
In the left navigation bar of the CloudStack UI, select Infrastructure. Under Clusters, click View All. Select the XenServer cluster and click Actions - Unmanage.
This may fail if there are hosts not in one of the states Up, Down, Disconnected, or Alert. You may need to fix that before unmanaging this cluster.
Wait until the status of the cluster has reached Unmanaged. Use the CloudStack UI to check on the status. When the cluster is in the unmanaged state, there is no connection to the hosts in the cluster.
To clean up the VLAN, log in to one XenServer host and run:
```
/opt/xensource/bin/cloud-clean-vlan.sh
```
Prepare the upgrade by running the following on one XenServer host:
```
/opt/xensource/bin/cloud-prepare-upgrade.sh
```
If you see a message like "can't eject CD", log in to the VM and umount the CD, then run this script again.
Upload the hotfix to the XenServer hosts. Always start with the Xen pool master, then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the hotfixes to the host. Place them in a temporary folder such as /root or /tmp.
On the Xen pool master, upload the hotfix with this command:
```
xe patch-upload file-name=XS602E003.xsupdate
```
Make a note of the output from this command, which is a UUID for the hotfix file. You'll need it in another step later.
Note
(Optional) If you are applying other hotfixes as well, you can repeat the commands in this section with the appropriate hotfix number. For example, XS602E004.xsupdate.
Manually live migrate all VMs on this host to another host. First, get a list of the VMs on this host:
```
# xe vm-list
```
Then use this command to migrate each VM. Replace the example host name and VM name with your own:
```
# xe vm-migrate live=true host=host-name vm=VM-name
```
Troubleshooting
If you see a message like "You attempted an operation on a VM which requires PV drivers to be installed but the drivers were not detected," run:
/opt/xensource/bin/make_migratable.sh b6cf79c8-02ee-050b-922f-49583d9f1a14.
Apply the hotfix. First, get the UUID of this host:
# xe host-list
Then use the following command to apply the hotfix. Replace the example host UUID with the current host ID, and replace the hotfix UUID with the output from the patch-upload command you ran on this machine earlier. You can also get the hotfix UUID by running xe patch-list.
xe patch-apply host-uuid=host-uuid uuid=hotfix-uuid

Copy the following files from the CloudStack Management Server to the host.

Copy from here...	...to here
`/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py`	`/opt/xensource/sm/NFSSR.py`
`/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/setupxenserver.sh`	`/opt/xensource/bin/setupxenserver.sh`
`/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/make_migratable.sh`	`/opt/xensource/bin/make_migratable.sh`

(Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud Support Pack.
- Download the CSP software onto the XenServer host from one of the following links:
  For hotfix XS602E005: http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz
  For hotfix XS602E007: http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz
- Extract the file:
  # tar xf xenserver-cloud-supp.tgz
- Run the following script:
  # xe-install-supplemental-pack xenserver-cloud-supp.iso
- If the XenServer host is part of a zone that uses basic networking, disable Open vSwitch (OVS):
  # xe-switch-network-backend bridge
Reboot this XenServer host.
Run the following:
/opt/xensource/bin/setupxenserver.sh
Note
If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or directory" appears, you can safely ignore it.
Run the following:
for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk '{print $NF}'`; do xe pbd-plug uuid=$pbd ;
On each slave host in the Xen pool, repeat these steps, starting from "manually live migrate VMs."

Chapter 5. API Changes in 4.1.1

5.1. New API commands in 4.1
5.2. Changed API commands in 4.1

5.1. New API commands in 4.1

CloudStack 4.1.1 is a maintenance release, and thus there are no new API commands.

5.2. Changed API commands in 4.1

CloudStack 4.1.1 is a maintenance release, and thus there are no new API commands.

Version 4.1.1 Release Notes

Apache CloudStack

Legal Notice

Abstract

Preface

1. Document Conventions

1.1. Typographic Conventions

1.2. Pull-quote Conventions

1.3. Notes and Warnings

Note

Important

Warning

2. Submitting Feedback and Getting Help

Chapter 1. Welcome to CloudStack 4.1

Chapter 2. Compatibility Matrix for 4.1.1

2.1. Supported Operating Systems

2.2. Supported Hypervisors

Bare Metal Support

2.3. Supported Browsers

2.4. External Devices

Chapter 3. Version 4.1.1

3.1. Issues Fixed in 4.1.1

3.2. Known Issues in 4.1.1

Chapter 4. Upgrade Instructions

4.1. Upgrade from 4.1.0 to 4.1.1

Community Packages

Community Packages

4.2. Upgrade from 4.0.x to 4.1.1

Package Structure Changes

Community Packages

Community Packages

For Xen Hosts: Copy vhd-utils

4.3. Upgrade from 3.0.2 to 4.1.1

Note

Community Packages

Community Packages

Note

Note

Note

Note

Note

Note

Troubleshooting

Note

Troubleshooting Tip

4.4. Upgrade from 2.2.14 to 4.1.1

KVM Hosts

Community Packages

Community Packages

Note

XenServer or KVM:

ESXi

Note

Troubleshooting

Note

Chapter 5. API Changes in 4.1.1

5.1. New API commands in 4.1

5.2. Changed API commands in 4.1