Enterprise Applications

Resource Sharing Across Users Resource Sharing Across Users

in Server Clusters in Server Clusters

Krithi Ramamritham IIT Bombay

krithi@iitb.ac.in

Optimizing and Scaling

Optimizing and Scaling

Enterprise Applications

Enterprise Applications

Enterprise (Information) Systems Enterprise (Information) Systems

• Any kind of computing system that is of "enterprise class"

– offering high quality of service

– dealing with large volumes of data

– capable of supporting a large organization -- “an enterprise“

• Enterprise Information Systems

– provide a technology platform that enables organisations to integrate and coordinate their business processes.

– provide a single system that is central to the organisation.

– ensure that information can be shared across all functional levels and management hierarchies.

– help eliminate the problem of information fragmentation caused by multiple information systems in an organisation.

• Enterprise applications are constructed using a multi-tier architecture for

simplified development and maintenance

• Considerable time and money is invested in the server infrastructure

• A significant amount of developer time is being spent to optimize Web applications

Web Server

Layer

Application Server

Layer

Database Request

Content

Optimizing and Scaling Enterprise Applications…

Optimizing and Scaling Enterprise Applications…

WS vs. AS WS vs. AS

• Web servers

– Do well defined and quantifiable local work

• e.g., processing HTTP headers, serving static content

• Application servers

– Run multi-layer programs

• e.g., scripts involving calls to backends

… …

Web Switch

Web Server Cluster

Application Server Cluster

… …

Web Switch

Web Server Cluster

Application Server Cluster

Inside the Application Layer Inside the Application Layer

3-tier model 3-tier model

PRESENTATION

BUSINESS LOGIC

DATA CONNECTOR

HTML

Objects

Row Set

• JDBC

• ODBC

• Servlets

• COM+

• EJB

• JSP

• ASP

Legacy Systems Databases

ADDT’L SERVICES

• Commerce

• Content Mgt.

• Personalization

Inside the Application Layer…

Inside the Application Layer…

PRESENTATION

BUSINESS LOGIC

DATA

CONNECTOR ^{• JDBC• ODBC}

Code

...

Block(s)

Code

...

Block(s)

Legacy Systems Databases

ADDT’L SERVICES

• Commerce

• Content Mgt.

• Personalization

1. JSP invokes a Servlet

2. Servlet contacts CMS

3. CMS requests data 4. DBMS calls

storage system

Performance and Scalability Performance and Scalability

Issues Issues

• Computationally-intensive logic executed at multiple tiers

• Cross-tier communication

• Object instantiation and cleanup processing

• External I/O calls

• Database connection pool latencies

• Content conversion and formatting

Optimizing the Application Layer Optimizing the Application Layer

Traditional Means Traditional Means

• Optimize each tier independently:

– Presentation-level caches built inside application server processes

– Main memory database employed over persistent DBMS – Persistent object storage techniques employed inside

content management systems … and so on

PRESENTATION

BUSINESS LOGIC

DATA

CONNECTOR ^{• JDBC• ODBC}

• Servlets

• COM+

• EJB

• JSP

• ASP

ADDT’L SERVICES

Local cache and optimization

code

Query result caching Query result caching

• Many application server products offer this feature

-- mitigates only local database access latency

-- only a subset of query results may be reused in page generation

-- page fragments may not all be from databases

Middle tier database caching Middle tier database caching

• Caching database tables in main memory Oracle 9i Cache

Main-memory databases, e.g., TimesTen

-- mitigates only database access latency

-- caching at table granularity results in poor cache utilization

-- main-memory databases are difficult to integrate

and maintain and can be expensive

Page Level Caching Page Level Caching

• Dynamically generated HTML pages are cached

+

-- increasing the risk of delivering incorrect pages – Often introduces excessive invalidations

-- e.g., even if a single element on the page changes

Optimizing the Application Layer Optimizing the Application Layer

Issues Issues

• Traditional techniques impact specific components within the application, but not the entire application

– No mitigation of component-to-component interaction latencies – Different synchronization and invalidation policies risk

data integrity

– Each optimization scheme consumes programmer time for development and maintenance

Key ideas Key ideas

• Re-use program results to eliminate redundant work

• Facilitate single-point, architecture-wide optimization

Apply to both

programmatic objects and result fragments

Optimizing the Application Layer Optimizing the Application Layer

PRESENTATION

BUSINESS LOGIC

DATA

CONNECTOR ^{• JDBC• ODBC}

• Servlets

• COM+

• EJB

• JSP

• ASP

Legacy Systems Databases

ADDT’L SERVICES

• Commerce

• Content Mgt.

• Personalization

cache

Enables the results of programs to be re-used.

Usually….

Usually….

Legacy Systems

1. JSP invokes a Servlet

PRESENTATION

BUSINESS LOGIC

DATA CONNECTOR

• JDBC

• ODBC

Code

...

Block(s)

Code

...

Block(s)

Databases

ADDT’L SERVICES

• Commerce

• Content Mgt.

• Personalization

2. Servlet contacts CMS

3. CMS requests data 4. DBMS calls

storage system

Plus, at each step there are communication delays and logic processing delays

With Our Solution…

With Our Solution…

PRESENTATION

BUSINESS LOGIC

DATA

CONNECTOR ^{• JDBC• ODBC}

Code

...

Block(s)

Code

...

Block(s)

Function Parameter(s) Result

Real-time storage engine

Tags trigger calls to the storage engine.

Can store any program output, but is most commonly an HTML fragment or a Programmatic Object.

Chutney tags

When the Result of a

Function with a specific Parameter set is already known (and up-to-

date), the work normally necessary to produce that Result is bypassed.

Appl. Programming Interface

Cache Management Cache Management

• A critical aspect of any caching solution

• Support novel cache management strategies:

– Prediction-based cache replacement – Observation-based cache invalidation

Cache Replacement Cache Replacement

• Prediction-based replacement

⁻ fragments having lowest

probability of access replaced

⁻ Least-Likely-to-be-Used (LLU) – Access probabilities based on:

• Current user navigational patterns over site graph

(in the form of clickstreams)

• Historical user navigational patterns over site graph

(in the form of association rules)

News

Sports

Hockey

Schedules Scores

Players Teams

Site Graph

(News, Sports, Hockey)  Schedules = 20%

(News, Sports, Hockey)  Players = 15%

(News, Sports, Hockey)  Teams = 10%

(News, Sports, Hockey)  Scores = 55%

LLU

Cache Invalidation Cache Invalidation

Need to support common cache invalidation techniques:

– Time-based: Each cache element assigned a TTL – Event-based: Updates to the database send an

invalidation message to the cache

– On demand: Manual invalidation of selected elements

– ….

Cache Invalidation…

Cache Invalidation…

• Other invalidation techniques supported:

– Observation-based

• User-initiated updates are observed in scripts;

each such update sends an invalidation message to the cache

• Most appropriate for auction sites, online trading sites

• Invalidation does not require communication with the databases

– Keyword-based:

• Elements can be associated with keywords; e.g., a retailer may wish to invalidate all “seasonal”

items

– Regular expression-based:

• Elements can be invalidated based on regular expression matching

Other Fragment Level Caching…

Other Fragment Level Caching…

+

can offload presentation layer tasks

– runs in the application server process space => competes for server resources

– application server cluster

=> multiple cache instances, duplication of content,

additional synchronization overhead

app servers (e.g., BEA’s WebLogic, IBM’s WebSphere) cache fragments produced by JSP scripts

Application Server Cluster

Performance Study…

Performance Study…

Test Site

– Fictitious online retail site, allows browsing of product catalog

– Pages generated using JSP scripts

– Site content stored in Oracle database

– Database schema based on Dublin Core Metadata Open Standard

– Contains 200,000 products and 44,000 categories

– Each page consists of 3 components, each involving a database call

Performance Study…

Performance Study…

Test Setup

– Content Database Server:

Oracle 8.1.6

– Web/Application Server:

WebLogic 6.0 running on cluster of 2 machines – Server machines:

have 1 GB RAM, dual P III-933 Mhz processors run Windows 2K Advanced Server

Testing Methodology...

Testing Methodology...

• Baseline Parameters:

– Cache Size, i.e., percentage of fragments that fit into cache: 75%

– Cache replacement policy: LLU

• User load is varied by sending requests from client machines running Radview’s WebLoad

• Simulated users navigate site according to Zipf 80-20 distribution (i.e., 80% of users follow 20% of navigation links)

Performance Impact Performance Impact

80% faster response times through existing application infrastructure

Source: Fortune 100 client results

0 10 20 30 40 50 60

0 100 200 300 400 500

Number of Users

Average Response Time (seconds)

non-Chutney Chutney

Chutney Throughput Impact Chutney Throughput Impact

250% increase in transaction rates

Source: Fortune 100 client results

0 100 200 300 400 500 600 700

0 100 200 300 400 500

Number of Users

Transactions Per Second

non-Chutney Chutney

Broad Interoperability Broad Interoperability

Chutney cache Java-based

JSP, Servlets, EJB, BEA WebLogic, IBM WebSphere,

iPlanet, Broadvision, etc.

Microsoft-based ASP, COM, IIS, MS Transaction

Server, etc.

Other ColdFusion,

Perl, etc.

Multi-server, heterogeneous environments can interface with a single storage engine.

Presentation Business

Logic Data

Presentation Business

Logic Data

Presentation Business

Logic Data

Alternative: CDNs Alternative: CDNs

Sources

Repositories

Clients

Content Distribution

Networks

e.g., Akamai

Push Based Push Based

Core Infrastructure

Core Infrastructure

Request Distribution Request Distribution

within Clusters within Clusters

… …

Web Switch

ServerWeb Cluster

Application Server Cluster

… …

Web Switch

ServerWeb Cluster

Application Server Cluster

Maximizing affinity

Exploit application characteristics

Summary Summary

• Bottlenecks persist throughout multi-tier architectures

• Traditional optimization approaches focus on individual components, not the entire application

• Need a solution which optimizes every tier

of a web application, globally