[attachment=11874]
Advanced data management
Course purpose
 Course contents
 Lecturer Academic experience
 2006.9 ~2008.6 University of California, Irvine, Postdoc researcher
 2002.8 ~2006.8 National University of Singapore, PhD candidate
 1998.9 ~ 2001.1 Shanghai Jiao Tong University Master candidate
 Research in Postdoc
Course grading
 Any question and any comments ?
Cloud computing
 Why we use cloud computing?
 Why we use cloud computing?
Case 1:
Write a file
Save
Computer down, file is lost
Files are always stored in cloud, never lost
 Why we use cloud computing?
Case 2:
Use IE --- download, install, use
Use QQ --- download, install, use
Use C++ --- download, install, use
……
Get the serve from the cloud
What is cloud and cloud computing?
Cloud
Demand resources or services over Internet
scale and reliability of a data center.
Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a serve over the Internet.
Users need not have knowledge of, expertise in, or control over the technology infrastructure in the "cloud" that supports them.
 The architecture of cloud computing system
 Characteristics of cloud computing
 Virtual.
software, databases, Web servers, operating systems, storage and networking as virtual servers.
 On demand.
add and subtract processors, memory, network bandwidth, storage.
Software delivery model
 No hardware or software to manage
 Service delivered through a browser
 Customers use the service on demand
 Instant Scalability
Examples
 Your current CRM package is not managing the load or you simply don’t want to host it in-house. Use a SaaS provider such as Salesforce.com
 Your email is hosted on an exchange server in your office and it is very slow. Outsource this using Hosted Exchange.
Platform delivery model
 Platforms are built upon Infrastructure, which is expensive
 Estimating demand is not a science!
 Platform management is not fun!
Examples
 You need to host a large file (5Mb) on your website and make it available for 35,000 users for only two months duration. Use Cloud Front from Amazon.
 You want to start storage services on your network for a large number of files and you do not have the storage capacity…use Amazon S3.
Computer infrastructure delivery model
 A platform virtualization environment
 Computing resources, such as storing and processing capacity.
 Virtualization taken a step further
Examples
 You want to run a batch job but you don’t have the infrastructure necessary to run it in a timely manner. Use Amazon EC2
 You want to host a website, but only for a few days. Use Flexiscale.
 Cloud computing and other computing techniques
 An Industry Transformed
 Shrek, Delgo, and Others
 Grid Computing & Cloud Computing
 share a lot commonality intention, architecture and technology
 Difference
programming model, business model, compute model, applications, and Virtualization.
Grid Computing & Cloud Computing
 the problems are mostly the same
 manage large facilities;
 define methods by which consumers discover, request and use resources provided by the central facilities;
 implement the often highly parallel computations that execute on those resources.
Grid Computing & Cloud Computing
 Virtualization
 Grid
 do not rely on virtualization as much as Clouds do, each individual organization maintain full control of their resources
 Cloud
 an indispensable ingredient for almost every Cloud
Any question and any comments ?
 Google App Engine
 Google App Engine
 Does one thing well: running web apps
 Simple app configuration
 Scalable
 Secure
 App Engine Does One Thing Well
 App Engine handles HTTP(S) requests, nothing else
 Think RPC: request in, processing, response out
 Works well for the web and AJAX; also for other services
 App configuration is dead simple
 No performance tuning needed
 App Engine Architecture
 How to use Google App engine
 Download Java 6
 Download Eclipse and Google plug in
 Register a user account in Google
 Create an application (python, Java) and upload the code
 In class quiz
 Please answer all questions
 You may be requested to answer a question later. Your performance will affect your final score.

[attachment=12167]
Data Management:
What to do with all the data that organizations collect, store and use?
Information overload!
Solution
n Data warehousing
n Data access
n Data mining
n Online analytical processing (OLAP)
n Data visualization
n Data sources
4.3 The Nature and Sources of Data
n Data: Raw
n Information: Data organized to convey meaning
n Knowledge: Data items organized and processed to convey understanding, experience, accumulated learning, and expertise
DSS Data Items
– Documents
– Pictures
– Maps
– Sound
– Animation
– Video
Can be hard or soft
n Data Sources
n Internal
n External
n Personal
n 4.4 Data Collection and Data Problems
n Summarized in Table 4.1
4.5 The Internet and Commercial Database Services
For External Data
n The Internet: Major supplier of external data
n Commercial Data “Banks”: Sell access to specialized databases
Can add external data to the MSS in a timely manner and at a reasonable cost
n The Internet/Web and
Corporate Databases and
Systems
Use Web Browsers to
n Access vital information by employees and customers
n Implement executive information systems
n Implement group support systems (GSS)
Database management systems provide data in HTML
Data Management
n External data needs to be considered in making organizational decisions.
n Data security, quality, and integrity are critical.
n Selecting data management tools can be a major problem.
Data Life Cycle Process
1. New data collection occurs from various sources.
2. It is temporarily stored in a database then preprocessed to fit the format of the organizations data warehouse or data marts
3. Users then access the warehouse or data mart and take a copy of the needed data for analysis.
4. Analysis (looking for patterns) is done with
 Data analysis tools
 Data mining tools
n Data Life Cycle Process Continued
4.6 Database Management Systems in DSS
n DBMS: Software program for entering (or adding) information into a database; updating, deleting, manipulating, storing, and retrieving information
n A DBMS combined with a modeling language is a typical system development pair, used in constructing DSS or MSS
n DBMS are designed to handle large amounts of information
4.7 Database Organization and Structure
n Relational Databases
n Hierarchical Databases
n Network Databases
n Object-oriented Databases
n Multimedia-based Databases
Data and Applications
Traditional File Systems
n Advantages
– simple data design to support single or small group of applications
– fast data access
– inexpensive
Disadvantages
– lack of data relation
– redundancy
– lack of standards
– low application development productivity
– DBMS
Advantages
– integration, sharing of data
– increased data accessibility
– minimized redundancy
– easier application development and maintenance
– improved data security
– logical/physical data independence
Disadvantage
– complex data design
– slow access
– expensive
Relational Database
n Primary Key
– duplicate rows not allowed
– cannot have missing (NULL) value(s) for PK
n Foreign Key
– defines relationship between tables
– FK values either reference existing PK values or they are NULL
(referential integrity)
Relational Database
n Relational Operators
– Select: subset of rows
– Project: subset of columns
– Join: creates new table by linking on common attributes