CS 695
Topics in Virtualization and Cloud Computing
and Storage Systems
Hot or not?
source: Gartner Hype Cycle for Emerging Technologies, 2014
Cloud Computing
●
The in-vogue term
●
Cloud is the network
●
Network is the cloud
●
Cloud storage, storage cloud
●
Cloud-based e-learning
●
Application in cloud, cloud application
●
Smart cloud
●
Cloud and phones
What is cloud computing?
●
Definition?
●
Where is it?
●
Why the hoopla?
scale your infrastructure on demand within minutes or even seconds, instead of days or weeks, thereby avoiding under-utilization (idle servers) and over-utilization
'internet centric software.' ... shift from the traditional single tenant approach to software development to that of a scalable, multi-tenant, multi-platform, multi-network, and global a broad array of web-based services aimed at allowing users to obtain a wide range of functional capabilities on a 'pay-as-you-go' basis
…basically the broad concept of using the internet to allow people to access technology- enabled services ... services must be 'massively scalable' to qualify as true 'cloud
computing'
The 'cloud' model initially has focused on making the hardware layer consumable as on- demand compute and storage capacity.
Cloud computing really is accessing resources and services needed to perform functions with dynamically changing needs. ... The cloud is a virtualization of resources that maintains and manages itself.
outsourced, pay-as-you-go, on-demand, somewhere in the Internet, etc.
Cloud computing
●
Computing-as-a-service
●
All forms of computing
●
Analogies?
Cloud characteristics
●
On-demand service
– User not worried about maintenance and setup issues etc.
●
Shared Resources
– Large capacity distributed/multiplexed over several users
●
Flexible Provisioning
– Dynamically scale resources
●
Fine-grained metering
– pay-as-you-use model
Is the 'Cloud' brand new?
●
NO!
– 60s-70s, large centralized servers carved up for individual access
– Cluster computing
– Grid computing
– Utility computing
– Peaking now
● Decreasing cost of powerful hardware
● Virtualization
● Economics
Cloud service models
●
IaaS
– Infrastructure as a Service
– Amazon EC2, Ubuntu One, IBM Test Cloud
●
PaaS
– Platform as a service
– Google Appengine, Windows Azure
●
SaaS
– Software as a service
– Gmail, Google online services
●
*aaS
Cloud deployments
Cloud deployment options
●
Private cloud
– Infrastructure for a single organization
● Internal hosting, testing environment
●
Public cloud
– Infrastructure for general use
– Cloud provider owns service
●
Hybrid cloud
– Composition of private+public
– Secure/non-secure or QoS/best-effort traffic, Cloud bursting
Cloud views
●
User view
●
Provider view
User-view of the Cloud
●
On-demand provisioning
– Resource, software, services
– Quick deployment
– Security and isolation
●
SLA requirements
– Service-level agreement
– Availability, isolation, performance metrics ...
●
Pay-on-use model
– Minimize cost for service
Providers-view of the Cloud
●
Support on-demand provisioning
– Resource accounting and provisioning
●
Efficient provisioning
– Reactive and quick
– High multiplexing desired
– Meet SLA requirements of users
– Allocate minimum to maximize profit/#users
– Power!
● Administration and Manageability
● Data Privacy
● Data Services Architectures
● Distributed and Parallel Query Processing
● Energy Management
● Distributed and Cloud Networking
● High Availability and Reliability
● Infrastructure Technologies
● Large Scale Cloud Applications
● Multi-Tenancy
Scope of cloud computing
● Programming Models
● Provisioning and Metering
● Resource Management and Performance
● Scientific Data Management
● Security of Services
● Service Level Agreements
● Storage Architectures
● Transactional Models
● Virtualization Technologies
source: Call for papers ACM SOCC 2015
Virtualization
●
Didn't you mention “virtualization” in the course
title?
Begin from the beginning
●
Computer systems are incredibly complex
– silicon, transistors, disks, processors, network cards, buses ...
– device drivers, operating systems, compilers, application libraries ...
●
But still,
– Systems evolving and flourishing
– Why?
Abstractions and Interfaces
●
abstraction: concept or idea (w/o implementation specifics)
●
interface: operations/process related to abstraction
●
Complex systems organized as a hierarchy of abstractions
– Well-defined interfaces allow independent evolution of sub- systems
Abstraction+Interface Examples
Abstraction+Interface Examples
1.Storage
● Physical disk: tracks, sectors, disk head ...
● User view: file (abstraction)
● File operations (interface): OS + device driver
2.Network
3.ISA (Instruction Set Architecture)
● x86 architecture (8086 based, 16-bit)
● Intel, AMD, VIA ... manufacturers
● Software runs on all x86 hardware
Are we ready for world peace then?
●
What is the issue with abstraction oriented
development?
Are we ready for world peace then?
●
What's the issue with abstraction oriented development?
– Interoperability
– Sub-systems designed for one interface will not work for other
● e.g., binary incompatibility
Virtualization
●
virtual: not real, illusionary, perception of reality
●
virtualize a component/resource
●
Map visible interface/resources onto those of underlying physical system
●
real system appears virtual
Types of virtual machines
●
Process view of machine
●
memory, user-level instr., system calls for OS functions
●
OS interface to
hardware defines view of process
●
Process VM
●
System view of machine
●
environment to support multiple processes
●
sharing resources
●
hardware characteristics defines system view
●
System VM
Process VM and System VM
Virtualization Software Application
Operating System
Hardware Virtualization Software
Application
Operating System Hardware
Host Guest
Runtime
Process VM examples
Process VM examples
●
Multi-processing systems
●
Emulators
– Fetch, decode, interpret
– e.g., WINE, QEMU
●
High-level Language VMs
– Compile for virtual architecture
– Translate/execute on native platform
– JAVA, .Net
System VM characteristics
●
Provide machine-view (virtual machines) to multiple operating systems
– Resource allocations
● cpu, memory, network, disk
●
Control/arbitrate VMs access to physical hardware
●
Isolation and Security
System VMs
●
Native VMs (type 1)
– a.k.a Bare-metal VMs
– Hypervisor/VMM on hardware, controls guests directly
– Guest unaware of VM translation
●
Hosted VMs (type 2)
– Hypervisor/VMM hosted on OS
– Guests co-ordinated by VMM
– Para-virtualized or Fully-virtualized VMs
● Guest aware of underlying hypervisor?
Type of System VMs
●
Native VMs (Bare-metal)
●
VMware ESX
●
Hosted VMs
●
Xen
●
kvm
●
VMware GSX
●
VirtualBox
Virtualization benefits
●
Flexible resource provisioning
●
Increased Inter-operability
– Standard virtualized interface
●
Isolation
– Secure multiplexing, Local impact of crashes/bugs
– Resource usage isolation
●
Machine State Capture
– VM is a file
– Snapshots, cloning, migration
●
Content distribution
Problem Areas
Virtualization technolgies Capacity planning and profiling
Physical to virtual (p2v)
Placement and Provisioning Resource Management
Migration and Consolidation management
Network I/O optimizations in virtual machines
Storage Issues with virtualization
