+ All Categories
Home > Technology > Seminario Paolo Maggi, 24-05-2012

Seminario Paolo Maggi, 24-05-2012

Date post: 01-Sep-2014
Category:
Upload: crs4-research-center-in-sardinia
View: 749 times
Download: 0 times
Share this document with a friend
Description:
Il seminario mostra come le moderne tecnologie di Cloud Computing possono essere applicate per migliorare l'utilizzabilità e l'efficacia di strumenti di simulazione e di analisi di dati ambientali, con grandi vantaggi dal punto di vista della semplificazione dei flussi di lavoro.
Popular Tags:
46
Tecnologie Cloud per la fornitura di applicazioni Copyright NICE - 2012 ambientali HPC in modalità SaaS Paolo Maggi <[email protected]> R&D Manager
Transcript
Page 1: Seminario Paolo Maggi, 24-05-2012

Tecnologie Cloud per la fornitura di applicazioni

Copyright NICE - 2012

Tecnologie Cloud per la fornitura di applicazioni ambientali HPC in modalità SaaS

Paolo Maggi <[email protected]>R&D Manager

Page 2: Seminario Paolo Maggi, 24-05-2012

Tecnologie Cloud per la fornitura di applicazioni

Copyright NICE - 2012

Tecnologie Cloud per la fornitura di applicazioni ambientali HPC in modalità SaaS

High PerformanceComputing

SoftwareasaService

Page 3: Seminario Paolo Maggi, 24-05-2012

What is High Performance Computing?

� Refers to the use of supercomputers or clusters of computers to solve difficult computational problems that typically arise through scientific inquiry.

� Computer users turn to HPC when a problem is too large to solve on a conventional laptop or workstation or runs too slowly

Copyright NICE - 2012

runs too slowly– it requires too much memory or disk space– the algorithm is complex– the dataset is large– data access is slow

Page 4: Seminario Paolo Maggi, 24-05-2012

When we need HPC?

� To do a time-consuming operation in less time– I am an aircraft engineer– I need to run a simulation to test the stability of the wings at high

speed– I’d rather have the result in 5 minutes than in 5 days so that I

can complete the aircraft final design sooner

Copyright NICE - 2012

Page 5: Seminario Paolo Maggi, 24-05-2012

When we need HPC?

� To do an operation before a tighter deadline– I am a weather prediction agency– I am getting input from weather stations/sensors– I’d like to make the forecast for tomorrow before tomorrow

Copyright NICE - 2012

Page 6: Seminario Paolo Maggi, 24-05-2012

When we need HPC?

� To do a high number of operations per seconds– I am an engineer of ACME.com– My Web server gets 1,000 hits per seconds– I’d like my web server and my databases to handle 1,000

transactions per seconds so that customers do not experience bad delays

Copyright NICE - 2012

Page 7: Seminario Paolo Maggi, 24-05-2012

How to get my answer faster?

� Work harder– Get faster hardware

� Work smarter– Use optimized algorithms (libraries!)– Write faster code (adapt to match hardware)

Copyright NICE - 2012

– Write faster code (adapt to match hardware)– Trade convenience for performance (e.g. compiled program vs.

script program)

� Delegate parts of the work– Parallelize code– Use accelerators (GPGPU/CUDA)

Page 8: Seminario Paolo Maggi, 24-05-2012

Leveraging parallelism

Copyright NICE - 2012

Page 9: Seminario Paolo Maggi, 24-05-2012

Leveraging parallelism

Copyright NICE - 2012

Functional parallelismDifferent people are performing different tasks at the same time

Page 10: Seminario Paolo Maggi, 24-05-2012

Leveraging parallelism

Copyright NICE - 2012

Data parallelismDifferent people are performing the same task, but on different (equivalent) objects

Page 11: Seminario Paolo Maggi, 24-05-2012

What a cluster is…

� A cluster needs:– Several computers, nodes , often in special cases for easy

mounting in a rack– One or more networks (interconnections ) to hook the nodes

together– Software that allows the nodes to communicate with each other

(e.g. MPI)

Copyright NICE - 2012

(e.g. MPI)– Software, called Distributed Resource Manager (DRM), that

manage resources and assign them to individual users and computational jobs (e.g. LSF, PBS, SGE, …)

� A cluster is: all of those components working together to form one big computer (to give you answers in a fast way)

Page 12: Seminario Paolo Maggi, 24-05-2012

How a cluster looks like?

Copyright NICE - 2012

Page 13: Seminario Paolo Maggi, 24-05-2012

What is Cloud Computing?

Copyright NICE - 2012

Page 14: Seminario Paolo Maggi, 24-05-2012

A working definition of Cloud Computing ( NIST)

� Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications , and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

Copyright NICE - 2012

management effort or service provider interaction. � This cloud model promotes availability� Composed of

– five essential characteristics– three service models– four deployment models

Page 15: Seminario Paolo Maggi, 24-05-2012

The NIST Cloud definition framework

Service Software as a Platform as a Infrastructure as a

EssentialCharacteristics

Resource Pooling

Broad Network Access Rapid Elasticity

Measured Service

On Demand Self-Service

Copyright NICE - 2012

CommunityCommunityCloudCloud

Private Private CloudCloud

Public CloudPublic Cloud

Hybrid Clouds

DeploymentModels

ServiceModels

Software as a Service (SaaS)

Platform as a Service (PaaS)

Infrastructure as a Service (IaaS)

Page 16: Seminario Paolo Maggi, 24-05-2012

Cloud Computing Offerings

Copyright NICE - 2012

Page 17: Seminario Paolo Maggi, 24-05-2012

About NICE

� Pioneer in Technical and Engineering Cloud solutions� 14 years experience with enterprise

Grid/Cloud solutions throughout all industries

� Worked closely with ISV and HW vendors since the beginning

� Strong relationship with Research and Academia

Headquarters: Italy

Offices: USA, Germany

Copyright NICE - 2012

� Core business: Access to Grid / HPC solutions and Remote Visualization� EnginFrame Grid Portal product line

� DCV Remote Visualization technology

� Other relevant competencies � Distributed Resource Management

� Grid Intelligence

� Visualization technology integration

Page 18: Seminario Paolo Maggi, 24-05-2012

NICE Customers and Market Segments

Automotive & Industrial Equipment

Audi, ARRK, Bridgestone, Bosch, Daimler, Delphi, Dow Chemical, FIAT, Ferrari, Honda,

Energy

Addax Petroleum, AECL, Hess, Bayerngas, BHP Billiton, Beicip, British Gas, Centrica, Chevron, Conoco-Phillips, Dong, Dowell, DSC-Libya, ENI/Agip, GazPromNeft, GDF, Logelco, Maersk Oil, Marathon Oil, Nexen, Novatek, Papuan Oil, Rosneft, Schlumberger, Shengli Oil, Sonatrach, Statoil, Talisman Energy, TNK-BP, TNNC, TOTAL, WesternGeco, Xinjiang Oil

Aerospace & Manufacturing

AIRBUS, Air Products and Chemicals, Procter&Gamble, SelexGalileo, Goodrich Aerospace, Hamilton Sunstrand, Kimberly Clarke, Magellan Aerospace, Nordam, Northrop Grumman, Raytheon, Thales

Copyright NICE - 2012

Delphi, Dow Chemical, FIAT, Ferrari, Honda, Hyundai, Jaguar-Land Rover, Lear, Magneti Marelli, McLaren, P+Z, PSA, Tata Steel, Toyota, TRW, Volkswagen

Life Sciences

Bayer, Biolab, DEISA project, Swiss Institute for Bioinformatics, Partners Healthcare, Pharsight, M.D. Anderson Cancer Center

TOTAL, WesternGeco, Xinjiang Oil

Research & Education

ASSC, Beihang Uni, CCLRC, CILEA, CNR, CNRS/IN2P3, ENEA, Georgia State Uni, INFN, RMSC, Harvard Uni, Messina Uni, Huazhong Normal Uni, Yale Uni

Others

Accent, Samsung SDI, SensorDynamics, Bank of Italy, Deutsche Bank

Page 19: Seminario Paolo Maggi, 24-05-2012

The NICE Grand Vision

InternalResources

Administrators& Managers

Supplier

Partner

Copyright NICE - 2012

Customer

HPC Centers, Public Clouds,

User

Multi-Tenancy &Collaboration services

Page 20: Seminario Paolo Maggi, 24-05-2012

NICE Products & Solutions

�� NICE EnginFrameNICE EnginFrame

�� NICE DCVNICE DCV

Copyright NICE - 2012

�� NICE EnginFrame ViewsNICE EnginFrame Views

�� Solutions for vertical marketsSolutions for vertical markets:

� Life Science

� Oil&Gas

� Automotive

Page 21: Seminario Paolo Maggi, 24-05-2012

NICE Products Overview

Service Service CC

((SaaSSaaS))

Service Service CC

((SaaSSaaS))

Service A Service A

(HPC)(HPC)

Service A Service A

(HPC)(HPC)Service Service XX

((workflowworkflow))

Service Service XX

((workflowworkflow)). . .Service BService B

(3D application)(3D application)

Service BService B

(3D application)(3D application)

Customer 1 (Corporate network) Customer N. . .Customer 2 (VPN)

Vertical solutions

Service Presentation Layer(Web, API, Command-line, ...)

Project Mgr End Users Developers End UsersDevelopers End Users

Access ControlAD, LDAP, NIS, …

Copyright NICE - 2012

GridMLdriver

CloudMLdriver

ViewsMLdriver

VMWARE, EC2, MOAB, ISF, …

DCV, VNC, RGS, TAW, …

LSF, Lava, SGE, PBS, MOAB, …

Infrastructure model abstraction

Desktop CloudVisualization

Vertical solutions

DataGateSLB-RE

…HPC

Views

Accounting & Billing

RDBMS

Page 22: Seminario Paolo Maggi, 24-05-2012

Linux & Windows

3D sessions

Our architecture

Collaborators,Support staff

Self-Service Offering

End Users

Thin viewer

ResourcesAccess

VisualizationServers

HTTP(S)H

PC

schdulers

DCV protocol

Copyright NICE - 2012

End Users schdulers

HPC jobs

Command-line

Developers, Integrators

SOAP

Orchestration &

P

rovisioning

Storage

Page 23: Seminario Paolo Maggi, 24-05-2012

User Experience in the Technical CloudOpen Grid

Portal

CommercialHPC Cloud

HPC

Enterprise Grid

Desktop Scavenging

Copyright NICE - 2012

HPC Clusters

Page 24: Seminario Paolo Maggi, 24-05-2012

NICE EnginFrame key features

� Lightweight Web front-end for HPC systems� Best in class data management� WebServices API and command-line interface� Controlled, user friendly application submission

Supports all mainstream job schedulers

Copyright NICE - 2012

� Supports all mainstream job schedulers� Workload, resource, license monitoring

Page 25: Seminario Paolo Maggi, 24-05-2012

Batch job / workflow submission

User friendly,Application-oriented

Job submission

Copyright NICE - 2012

Flexible and efficientInput file management

Hide complexity ofUnderlying scheduler

Page 26: Seminario Paolo Maggi, 24-05-2012

Application Data Management

Application data can be organized into projects

Application data can be marked as starred

Copyright NICE - 2012

starred

Metadata can be associated to application data

Page 27: Seminario Paolo Maggi, 24-05-2012

Monitoring

Jobs, Hosts, Queues, Licenses, …

Copyright NICE - 2012

Page 28: Seminario Paolo Maggi, 24-05-2012

Data transfers & file management

The file manager component allows to seamless navigate and access server-side files from the web browser

Copyright NICE - 2012

Page 29: Seminario Paolo Maggi, 24-05-2012

SOA-enabled job submission

� WS-I interface� Java / .NET client library and

command line interface� Simplifies integration with

client-side applications (optimization, workflow, etc.) for power-users

Copyright NICE - 2012

Page 30: Seminario Paolo Maggi, 24-05-2012

What is a 3D Cloud?

� "Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications , and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.” (NIST)

Copyright NICE - 2012

management effort or service provider interaction.” (NIST)

� 3D Clouds will enable on-demand network access to interactive 3D applications (like visualization applications for scientific data, CAD applications, etc.)

Why NICE customers need a 3D Cloud?

Page 31: Seminario Paolo Maggi, 24-05-2012

A typical scenario in the CAE computing sector

1 - Simulation submission

Win

2 - Data Transfer to visualize results

Copyright NICE - 2012

Server/Data Center

Win

2 - Data Transfer to visualize results

3 - CollaborationNeeds Data exchange CAE Workstations

• Can run small to medium serial analysis• Requires high-end GPU for rendering• May represent IT management

challenges (tens to hundreds of seats)

Page 32: Seminario Paolo Maggi, 24-05-2012

Problems associated to this model

NETWORKNetwork overload leading to poor performance and response times all round

COSTExpensive, dedicated workstations (GPU, memory, …) with short lifecycle

IT MANAGEMENTSupport, update and replace tens to tens of thousands of workstations

Copyright NICE - 2012

WORKSTATION SIZINGWorkstations have to be sized for the largest expected models

SECURITYMoving sensitive data around (in/out organization) is always risky

WORKFORCECurrent models do not support a diverse mobile workforce

Page 33: Seminario Paolo Maggi, 24-05-2012

Collaboration

Moving technical applications closer to the data

PCWS

HPC job

submission and

management

HPC job

submission and

management

Remote

Visualization

Remote

Visualization

Copyright NICE - 2012 33

Linux

Job submission

managementmanagement VisualizationVisualization

Page 34: Seminario Paolo Maggi, 24-05-2012

Benefits of the HPC Cloud model

NETWORKNetwork is no more a bottleneck and data loads faster

COSTCentralized & Shared servers are less expensive to buy & manage

IT MANAGEMENTSupport, update and replacement are more efficient & do not affect users

Copyright NICE - 2012

WORKSTATION SIZINGResources are dynamically sized based on users needs

SECURITYSensitive data remain within protected data center with full access control

WORKFORCEUsers can virtually connect & collaborate from anywhere with any client

Page 35: Seminario Paolo Maggi, 24-05-2012

� Technologies providing secure remote network access to interactive 3D applications leveraging server-side graphic hardware acceleration (GPUs)

Enabling Technologies for the 3D Clouds

Copyright NICE - 2012

� A software stack that allows the end-user to easily launch and access remote interactive applications and takes care of managing and load balancing applications and desktop sessions running within a Visualization Farm

Page 36: Seminario Paolo Maggi, 24-05-2012

NICE EnginFrame Views

RealVNC

2D & 3D Applications2D & 3D Applications

Job Scheduler & Distributed Resource Manager

NICE EnginFrame (Portal and Gateway)

VirtualGL NICE DCV HP RGS NX, RDP, ...

Copyright NICE - 2012

RealVNC

Storage Infrastructure

VirtualGL NICE DCV HP RGS

Computing Infrastructure

LINUX

SOLARIS

WINDOWS

NX, RDP, ...

Page 37: Seminario Paolo Maggi, 24-05-2012

How Does It Work?

EnginFrame

1

The user connects to EnginFrame to create a new Session

2

Job sch. selects the appropriate node and starts the session

3

User gets connected to the session

Win

Copyright NICE - 2012

Job Sch.

64GBytes, Linux

8GB, Windows

4GB, Win/Lin blades

User gets connected to the session on the selected node

Heterogeneous infrastructure: HW, OS, middleHeterogeneous infrastructure: HW, OS, middle--wareswares

Page 38: Seminario Paolo Maggi, 24-05-2012

Benefits for the End Users

� Access to Applications as a Service– Hide infrastructure details (site, platform, etc…)– Session allocation can be influenced by memory requirements, data

location affinity and other customer-specific parameters– Intelligent load balancing of sessions, based on the Job Scheduler

• Memory-aware, Memory reservation, Application license-aware– Fewer or better optimized data transfers

� Collaboration, session sharing– The session owner can generate a “URL” that can be sent by email /

Copyright NICE - 2012

– The session owner can generate a “URL” that can be sent by email / instant message to invite a collegue to join a given session, without disclosing the user’s password

� Easy management of active session– Create new sessions with user-specific preferences (resolution, etc...)– List , reconnect and kill existing sessions

� Seamless Access to the sessions and Single Sign-On– Automation of session-level password create/destroy

• E.g. login via NTLM / ActiveDirectory credentials and map to Linux user

Page 39: Seminario Paolo Maggi, 24-05-2012

Benefits for the Administrators

� Increased level of service provided to users– Sessions are load balanced by the Job Scheduler to match user needs– Memory reservation and Data locality scheduling– Reduce help desk calls– Exposed services can be personalized per user/group/project

� Accounting– Sessions are jobs, so the resource usage accounting by user, group,

project can be collected through any Analytics toolMonitoring

Copyright NICE - 2012

� Monitoring– The load and usage of the login farm is monitored via EnginFrame– Node loading conditions, active sessions– Administrators can control and manage users’ idle or stuck sessions

� Support– Administrators can connect to user’s sessions to provide support

� Security– Easy integration into identity services, SSO, Enterprise portals

Page 40: Seminario Paolo Maggi, 24-05-2012

NICE DCV (Desktop Cloud Visualization)

� Originated in IBM Research in 2004, acquired by NICE in summer 2010

� Provides efficient remote access to graphic-intensive, professional OpenGL applications

� A central (graphics-enabled) server is accessed by remote users

Copyright NICE - 2012

� Users only need low-end machines with network connectivity to the server in order to view and interact with remote application

� One or more users (collaborators) can simultaneously access the server

� Is the key technology for delivery of real-time 3D graphics but relies on other software to provide the collaborative environment (RealVNC Visualization Edition)

Page 41: Seminario Paolo Maggi, 24-05-2012

NICE DCV – Main Features

� Works on Windows and Linux, leveraging high-end NVIDIA GPUs

� Support GPU sharing across multiple users, multiple OS

– First product on the market to allow the sharing of physical GPUs between multiple Windows VMs while maintaining full OpenGL acceleration and workstation-class performance

� Bandwidth optimized balancing of quality Vs. frame rate

Copyright NICE - 2012

� Bandwidth optimized balancing of quality Vs. frame rate

� High network latency tolerance

� Validated and optimized for Technical Computing applications

� Perfectly integrated with NICE EnginFrame for session management

Page 42: Seminario Paolo Maggi, 24-05-2012

Windows application delivery

Copyright NICE - 2012

Page 43: Seminario Paolo Maggi, 24-05-2012

Linux application delivery

43

Copyright NICE - 2012

Page 44: Seminario Paolo Maggi, 24-05-2012

Web-based Thin client access

Copyright NICE - 2012

Page 45: Seminario Paolo Maggi, 24-05-2012

Built-in collaboration45

Copyright NICE - 2012

Page 46: Seminario Paolo Maggi, 24-05-2012

Thank you!

Copyright NICE - 2012


Recommended