Showing posts tagged with virtualization

in project, space, control panel, virtualization, update, spacepanel

I've been pleasantly surprised with the amount of positive response I received after introducing Space in a previous post, so much so that it really inspired me to keep working on the project.

Over the past week, I've been living and breathing Space, and I am very happy to say I have made a great deal of progress.

New Space Dashboard

I'd like to take a moment to go over some of the biggest changes:

  • The UI - The old UI was just plain on Bootstrap and it looked pretty bad in my opinion. I decided to revamp this and make this project look like something out of 2015. The image above is the dashboard, below is the new server list:

New Server List

  • With that, I'm certain you've spotted the next major feature - DigitalOcean and Linode support. You can now add in your API key for either service provider and create Droplets and or Linodes right from Space.

  • Lastly, the installation scripts have been improved (in that they actually work now). There are some issues here and there, but they can resolved fairly easily. In addition, I rewrote the README to include more useful information about problems you may have while installing and using Space.

I'd also like to note I've made a website for this project over at https://spacepanel.io. In the future I would like to move announcements to the Space website directly, but for the time being I'll continue making them here.

As always, if you have any trouble with Space, or if you have questions, comments or concerns please don't hesitate to reach out to me. Thanks for reading!

in python, space, virtualization, libvirt, development

When I first started working on Space I knew I would need to use something like libvirt to make all of the calls to the hypervisor its self. I had heard of libvirt several times during my tenure in the IT industry so I figured it would be realtively easy to get things running fairly quickly but when I started reading the libvirt documentation I was very surprised at how poor it was.

I'm not hoping to rewrite all of their documentation, but I would like to give a few examples for doing basic things using the libvirt API in Python.

Connecting to the Hypervisor

You'll need to establish a connection with your hypervisor before you can interact with it. A simple function to build that connection looks like this:

import libvirt

def connect():
	connection = libvirt.open("SYSTEM_TYPE:///system")
    return conn

You'll end up returning a connection object that you can then use to do other things. SYSTEM_TYPE will be something like QEMU or XEN.

Getting a List of Domains (Virtual Machines)

There are actually two calls required to do this, listDefinedDOmains() which returns a list of all defined domains (running and not running), and listDomainsID() which returns a list of only running domains.

import libvirt

connection = libvirt.open("SYSTEM_TYPE:///system")

# This lists all domains on the host
connection.listDefinedDomains()
# Returns
['vm1','vm2','vm3']

# This lists running domains on the host
connection.listDomainsID()
# Returns
[2] 
# These are the ID's as used by the hypervisor - you'll need to store this somewhere to tie this to a virtual machine as it will change each time the virtual machine is started.

Find a Domain

You can find a domain two different ways, you can use the ID that was mentioned above (this changes, so it's tough to use this unless you maintain this information somewhere) or you can use the name of the virtual machine.

I generally use the name because I set it and it stays the same regardless of what happens to the virtual machine. The name is what you'll see if you run virsh list --all:

 virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     vm54d42cd2a5ee223e6be95bc2     running

So you can do that using the following functions:

# Assume the connection function is included here

virtual_machine = connection.lookupByName("vm54d42cd2a5ee223e6be95bc2")
# Returns the object representing this virtual machine

virtual_machine = connection.lookupByID(2)
# Returns the same object, but uses the ID as shown in the virsh output. Probably not the one you want to use, but it is there if you need it.

Start Domain

Once you have the domain object you can start/stop/redfine it fairly easily. Let's look at starting it:

virtual_machine.create()
# Will create (start) this virtual machine if it is defined
Stop Domain
virtual_machine.destroy()
# Will destroy (stop) this virtual machine if it is defined

Create Domain

Now lets make a brand-spankin'-new domain. This wil take a bit of work, so lets dig in.

First you'll need to make a domain configuration file - this is an XML file that defines a bunch of things about the domain. I've provided an example below:

<domain type="kvm">
    <name>vm11</name>
    <cpu>
        <topology cores="4" sockets="1" threads="4" />
    </cpu>
    <uuid>90e7bca4-97b2-11e4-86bf-001e682ee78a</uuid>
    <memory unit="MB">512</memory>
    <currentMemory unit="MB">512</currentMemory>
    <vcpu placement="static">2</vcpu>
    <os>
        <type>hvm</type>
        <boot dev="hd" />
    </os>
    <features>
        <acpi />
        <apic />
        <pae />
    </features>
    <clock offset="utc" />
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>restart</on_crash>
    <devices>
        <emulator>/usr/libexec/qemu-kvm</emulator>
        <disk device="disk" type="file">
            <driver cache="none" name="qemu" type="raw" />
            <source file="/var/disks/vm11.img" />
            <target dev="hda" />
            <address bus="0" controller="0" target="0" type="drive" unit="0" />
        </disk>
        <disk device="cdrom" type="file">
            <source file="/var/images/ubuntu-14.04.1-server-amd64.iso" />
            <driver name="qemu" type="raw" />
            <target bus="ide" dev="hdc" />
            <readyonly />
            <address bus="1" controller="0" target="0" type="drive" unit="0" />
        </disk>
        <interface type="network">
            <source network="default" />
        </interface>
        <graphics port="-1" type="vnc" />
    </devices>
</domain>

I won't dig into how this is made (perhaps I'll do that in another post), but long story short you'll need to make this file and save it somewhere on your filesystem. You can take a look here to see how I did this in Space.

Next, we'll need to use this configuration to define a new virtual machine. Defining a virtual machine essentially means you are creating it, once it is defined you can undefine it (delete it), start it, and shut it down. To define the domain, you can use defineXML(xml):

# First you'll want to get your XML file into memory
xml = ""
with open(path_to_xml_config, "r") as file:
	xml = file.read()
connection.defineXML(xml)
# If you don't get an error, the domain has been created, woot!

Delete Domain

Deleting a domain is significantly easier than making one. You'll just need to use the undefine() function:

virtual_machine = connection.lookupByName("your_domain_name")
virtual_machine.undefine()
# Bye bye virtual machine, its dead! 

Updating Domain

I'm sure some of you have put two and two together here and have pieced together how you would update a domain from that - if not, I'll detail that now.

You'll first need to overwrite the old configuration file (or make a new one). The new configuration file should include the new options you want to update the domain to. Next, you'll need to undefine the current domain using the method described above.

Once the domain has been undefined, simply define it again as described in the create a domain section. Start it up and you'll have a domain with updated settings.

Conclusion

I hope this saves somebody time in the future because it took me a while to figure out how to get all this stuff working my first time around. If you have any questions about anything I didn't cover, please feel free to comment or hit me up on Twitter @joseph_pettit.

in space, control panel, virtualization, celery, mongodb, kvm, libvirt, spacepanel

One of my goals for 2015 was to learn more about virtualization - for me, learning requires a little reading and a ton of doing. I'd like to introduce my latest project that is a result of the aforementioned "doing" - Space.

Space is a virtualization control panel similar to cPanel, minus ripping out the guts, heart and soul of your operating system. Space is built in Flask and utilizes some new technologies for me, including libvirt, KVM, Celery and MongoDB.

Screen shot of Space Panel

Space is a web based GUI for libvirt essentially, meaning it has all of the flexibility offered by libvirt (in theory). Currently it runs on Centos 6.6 and KVM only, but I plan on expanding that to Debian/Ubuntu and Xen in the coming weeks/months/*.

You can do nearly everything you would need to do to manage a virtual machine directly in Space, including:

  • Create new virtual machines
  • Delete virutal machines
  • Start/stop/restart VMs
  • Access virtual machine via web console
  • Resize disks
  • Manage disk images
  • Manage networking
  • Access logs, see events

I'm making Space freely available to anybody who wants to use it, the source is available on Github. Feel free to git clone and give it a whirl. Be warned though, there are still some bugs that need to be worked out, so I wouldn't recommend using this in any environment even closely resembling production.