Spotlight on the petroleum and energy vertical
Download as PPT, PDF•1 like•620 views
The document discusses the challenges of managing big data in the petroleum and energy sector, emphasizing the need for efficient file transfer solutions. It introduces the integration of Enginframe and FileCatalyst to accelerate data transfer, reduce network congestion, and improve workflows for engineers submitting jobs across various global locations. A case study illustrates the significant time savings and enhanced reliability achieved through this technology partnership.
Spotlight on the petroleum and energy vertical
- 1. Spotlight on the Petroleum and Energy Vertical February 18, 2014
- 2. Introduction • John Tkaczewski, President and Co-Founder • Andrea Rodolico, CTO and Co-Founder, NICE | © Copyright 2 FileCatalyst, 2014
- 3. Overview • Big Data Challenges Facing Energy Organizations • FileCatalyst - Why Accelerated File Transfer? • NICE Introduction • Joint Solution – EnginFrame and FileCatalyst • Case Study | © Copyright 3 FileCatalyst, 2014
- 4. Big Data Challenges • Global operations Vs. HPC centralization • Data acquisition for drilling and exploration • Engineering, construction, simulation data is VERY big but it is usually highly compressible • Data distribution to geographically dispersed interpretation and analysis locations • Diverse networks (Fiber, Satellite, 3/4G, Cable/DSL, Microwave), sometimes unreliable or congested | © Copyright 4 FileCatalyst, 2014
- 5. Why Accelerated File Transfer? • File transfer acceleration technology allows large “big data” files to be sent quickly over large geographical distances without being impeded by network impairments such as latency and packet loss • Accomplished by transferring files using the UDP protocol instead of the TCP protocol, the protocol used for FTP • UDP wastes no time communicating about the receipt of a block of data so that regardless of network latency data transmission remains constant, thus overcoming the peak and valley effect that occurs with TCP based protocols on links with high latency • Reliability and congestion control is added to UDP at the application layer thus it doesn’t sacrifice its other desirable properties • Fine grain bandwidth management • Smart resume and re-try of large file transfers. MD5 checksum is performed before a file is resumed | © Copyright 5 FileCatalyst, 2014
- 6. Speed Gains with Acceleration vs. FTP | © Copyright 6 FileCatalyst, 2013
- 7. NICE Introduction • EnginFrame: the leading HPC Portal for Oil&Gas applications – Web based, intuitive, scheduler independent interface to address all HPC and Interactive needs of Technical Computing users – Works with leading HPC applications including reservoir simulation, seismic analysis, fluid dynamics, structural analysis, etc. • Desktop Cloud Visualization (DCV): 3D without workstations – Enable high end 3D applications to be run securely, close to the data, in the Private or Public Cloud, optimizing network usage over WAN and enabling collaboration and full mobility • NICE Services: accelerate Technical Cloud adoption – Comprehensive service offering to analyze, implement and evolve Technical Cloud solutions based on NICE and 3rd party tools | © Copyright 7 FileCatalyst, 2013
- 8. Linux & Windows 3D sessions O&G “Technical Cloud” Architecture Collaborators, Support staff Access Resources Self-Service Offering End Users Thin viewer Visualization Servers HTTP(S) HPC schdulers HPC jobs Developers, Integrators SOAP DCV protocol Storage FC Hot Folder FileCatalyst protocol FileCatalyst Server
- 9. Result: Fully Centralized HPC Workflows
- 10. 5 Key benefits of the Technical Cloud o Comprehensive, engineer-friendly experience Web based, command line, seamless job and data correlation o Boosts existing data management Centralized, distributed, mixed, incremental transfers o Hide the complexity of distributed systems Multiple HPC schedulers, multiple clusters, batch and visualization o Compatible with Mainstream ISV applications Wrapping of HPC solvers, programmable APIs, full 3D management o Ready for today’s complex Oil&Gas requirements Standards based, SSO support, secure deployments, non-intrusive
- 11. Case Study: Global Oil Company The engineers submit jobs from KL, Perth and other locations to Houston. The data sets vary from 2GB to 5GB. They transfer such large files to Houston using Windows Explorer, submit to the cluster and then copy back the results. The engineers submit jobs from KL, Perth and other locations to Houston. The data sets vary from 2GB to 5GB. They transfer such large files to Houston using Windows Explorer, submit to the cluster and then copy back the results. Houston Data transfer can take hours and overload the corporate network Data transfer can take hours and overload the corporate network d duurriningg p peeaakk t timimee..
- 12. Improved User Workflow 1. User in KL/Perth submits ECLIPSE jobs through EnginFrame – Input files are uploaded by EnginFrame Multiple File Upload Applet through FileCatalyst – Uploaded files are cached by EnginFrame – INCLUDE/SLAVES statements are analyzed and uploaded automatically • Before upload occurs, the central cache is checked • Cache is global for all the EnginFrame users, to maximize time savings – FileCatalyst provides bandwidth control during peak hours 1. Job starts and results are incrementally copied back to KL/Perth, while the job is running – EnginFrame “job spooler” sync through FileCatalyst – Unattended; no action required by the user
- 13. The Solution: EnginFrame + FileCatalyst FileCatalyst accelerates and automates transfer of input and output files. It seamlessly integrates with EnginFrame and eliminates most manual steps previously required. The integration has considerably improved the ease of use and reliability of the workflows, shortening time for each HPC iteration and reducing network usage and congestion. FileCatalyst accelerates and automates transfer of input and output files. It seamlessly integrates with EnginFrame and eliminates most manual steps previously required. The integration has considerably improved the ease of use and reliability of the workflows, shortening time for each HPC iteration and reducing network usage and congestion. Houston Web browser FC Hot Folder Server, Agent, Cache . FC Server Data transfers went down Data transfers went down f rfroomm h hoouurrss t too s seeccoonnddss!!
- 14. How to contact us • For more information about NICE – http://www.nice-software.com – [email protected] • For more information about FileCatalyst – http://www.filecatalyst.com • Full case study on: – http://www.filecatalyst.com/nice-software-accelerated-and-managed-file-transfer-| © Copyright 14 FileCatalyst, 2014
- 15. Upcoming Events - Prime Time, Ottawa Westin Hotel (Feb. 19-21) - FileCatalyst User Forum, Canadian High Commission, London, UK (Feb. 24th) - BVE 2014, Excel, London, UK (Feb 25-26) - CABSAT 2014, Dubai World Trade Centre (March 11-13) - NAB Show, Las Vegas, Nevada (April 5-10) And of course don’t forget our monthly webinars... | © Copyright 15 FileCatalyst, 2014