Breaking the Speed Limit with WEKA: The World’s Fastest File System on top of Azure Hot Blob

Abstract Azure Blob Storage is engineered to manage immense volumes of unstructured data efficiently. While utilizing Blob Storage for High-Performance Computing (HPC) tasks presents numerous benefits, including scalability and cost-effectiveness, it also introduces specific challenges. Key among these challenges are data access latency and the potential for performance decline in…

19/06/2024Azure High Performance Computing (HPC) Blog

Share:

You may be interested in

Product Announcement: Open OnDemand Integration with Azure CycleCloud Workspace for Slurm
Azure High Performance Computing (HPC) Blog,
We are thrilled to announce the preview of Open OnDemand integration with Azure CycleCloud Workspace for Slurm. Open OnDemand, developed by the Ohio Supercomputer Center (OSC), is an open-source, web-based…
27/03/2025
Achieving Optimal Performance for DeepSeek Expert Parallelism (DeepEP) on Azure
Azure High Performance Computing (HPC) Blog,
DeepEP DeepEP is a high-performance communication library developed by DeepSeek AI to optimize Mixture-of-Experts (MoE) and expert parallelism (EP) in large-scale AI models. It provides high-throughput, low-latency all-to-all GPU kernels for…
17/05/2025
Running AutoDock HPC application on Azure Machine Learning platform
Azure High Performance Computing (HPC) Blog,
Azure Machine Learning is a service that enables you to create, train, deploy, and manage machine learning models and experiments. You can also use it to run HPC applications, such…
03/01/2024
Azure Managed Lustre with Automatic Synchronisation to Azure BLOB Storage
Azure High Performance Computing (HPC) Blog,
Introduction This blog post walks through how to setup an Azure Managed Lustre Filesystem (AMLFS) that will automatically synchronise to an Azure BLOB Storage container. The synchronisation is achieved using the Lustre…
30/11/2023
Integrating Azure Managed Lustre Filesystem (AMLFS) into CycleCloud HPC Cluster
Azure High Performance Computing (HPC) Blog,
Overview: This blog discusses how easily we can integrate Azure Managed Lustre Filesystem into CycleCloud HPC cluster using a custom project named cyclecloud-amlfs. Azure Managed Lustre delivers the time-tested Lustre file…
14/07/2023
HPC Lift and Shift Cloud Migration: Architecture and Best Practices
Azure High Performance Computing (HPC) Blog,
Have you ever wondered what it takes to have an enterprise level HPC environment in the Cloud? What components should be in place and what steps should be taken to…
01/10/2024
Discover how Azure HPC can improve patient outcomes
Azure High Performance Computing (HPC) Blog,
Download the eBook 1McKinsey & Company, The next frontier of healthcare delivery, 2022.2Sesterce, Unleashing the Power of HPC in Healthcare: A Digital Odyssey, 2024. 3Accenture, Unleashing the power…
17/09/2024
Experience Next-Gen HPC Innovation: AMD Lab Empowers ‘Try Before You Buy’ on Azure
Azure High Performance Computing (HPC) Blog,
In today’s fast-paced digital landscape, High-Performance Computing (HPC) is a critical engine powering innovation across industries—from automotive and aerospace to energy and manufacturing. To keep pace with escalating performance demands…
11/03/2025
Annual Roundup of AI Infrastructure Breakthroughs for 2023
Azure High Performance Computing (HPC) Blog,
What a difference a year makes! Last year I said 2022 was a banner year for AI developments… if 2022 was a banner year then how should we describe 2023?…
27/03/2024
Microsoft at SC23
Azure High Performance Computing (HPC) Blog,
Come visit Microsoft at Supercomputing 2023 (SC23) November 12 – 17 where we’ll deep dive into high-performance computing (HPC) and AI solutions during an exciting week of sessions, hands on experiences and…
27/10/2023
Announcing the release of CycleCloud 8.6
Azure High Performance Computing (HPC) Blog,
CycleCloud 8.6: What's New and How to Get Started Learn about the latest features and enhancements of CycleCloud, the leading cloud HPC orchestration platform. CycleCloud is a powerful tool that…
29/02/2024
Automated deployment of CycleCloud and SLURM using CLI
Azure High Performance Computing (HPC) Blog,
Azure CycleCloud allows the creation of resources to run High Performance Computing (HPC) applications based on widely used job schedulers such as PBS, SLURM, and LSF. Once CycleCloud is installed,…
04/10/2023