HDInsight

Azure HDInsight has made notable improvements in stability and latency on Autoscale.

Apache Spark™ 3.3 on HDInsight is now in preview.

HDInsight is now generally available in Poland Central. 

Azure HDInsight for Apache Spark 2.4 will be retired as of February 10, 2024.

Azure HDInsight for Apache Kafka 3.2.0 now available for public preview and ready for production workloads.

Azure HDInsight for Apache HBase 2.4.11 now available for public preview and ready for production workloads.

Transition to the new Azure 'HDInsight API 2021-06-01' before 30 November 2024.

HDInsight is now available in more European regions, Sweden Central.

You can now create Azure HDInsight with restricted public connectivity resulting in a more secure networking infrastructure.

Enhance your security posture by leveraging Azure Private Link to connect to your HDInsight cluster privately across networks over Microsoft backbone network

HDInsight is now generally available for Australia Central customers.

HDInsight is now generally available for West US 3 customers.

Apache Spark 3.0 public preview in HDInsight 4.0 is a major update to Apache Spark and brings significant improvements including better performance and better support for ANSI SQL.

The Eav4-series virtual machines are ideal for memory intensive workloads, and offer great performance for large in-memory business critical workloads at competitive price points on Azure HDInsight.

HDInsight is now generally available for Norway East customers.

Azure HDInsight Kafka REST proxy greatly simplifies messaging architecture patterns by allowing Kafka producers and consumers to be located outside the virtual network and enables working with unsupported languages.

Adopt Dav4-series VMs in HDInsight to get great performance at a competitive price.

Azure HDInsight Internet Protocol Security (IPSec) encryption in transit is now generally available. HDInsight also supports version-less keys for customer-managed key (CMK) encryption at rest.

With Premium Azure Data Lake Storage Gen2 support for accelerated writes HDInsight HBase cluster, customers can now have increased read performance compared to standard storage and also take advantage of Hierarchical…

HDInsight ID Broker (HIB) provides modern OAuth authentication and enables federated users to login to cluster without syncing on-premise password hashes to the cloud.

Azure HDInsight private link integration allows you to create VNET injected clusters with no public IP and access them using your own private endpoints.

Azure HDinsight is now generally available in UAE Central

Enable HDInsight Autoscale to drive higher utilization of your clusters.

Azure HDInsight now supports virtual network service endpoint policies, which allow you to filter HDInsight egress virtual network traffic to certain Azure storage accounts.

HDInsight now fully supports distributions of Apache analytics projects in-house. Plus, in the coming months, we’re launching the next generation of HDInsight with new and improved features.

Three new security feature improvements are available for Azure HDinsight.

This new release enables you to perform Jupyter-like notebook operations and boosts collaborations with one-click conversion between IPYNB and PY files.

Azure HDInsight enterprise security enhancements include support for customer-managed key encryption with Azure Key Vault and a new feature, ID broker, which simplifies authentication setups.

Microsoft now fully supports distributions of Apache analytics projects in-house. Also, we’ll soon launch the next generation of HDInsight with improved provisioning and management experience, reserved instance pricing, low-priority virtual…

Starting on June 30, 2020, Azure HDInsight will enforce TLS 1.2 or later versions for all HTTPS connections. We recommend that you ensure that all your clients are ready to…

Azure HDInsight is now available in the UAE North region.

Beginning April 6, 2020, customers will be able to choose only 4-core or higher virtual machines (VMs) as head nodes for the new HDInsight clusters. Existing clusters will continue to…

Azure HDInsight now supports the use of service tags to simplify inbound network security group (NSG) rule management. Azure service tags group multiple IP addresses under a single user friendly…

Autoscale for Azure HDInsight is now generally available across all regions for Apache Spark and Hadoop workloads. This feature makes it possible to operate your big data analytics workloads in…

The accelerated writes feature in the HDInsight HBase cluster attaches a Premium SSD managed disk to every region server (worker node). Write ahead logs are configured to be written to…

A new Azure HDInsight capability enables you to customize and upgrade your Azure SQL Database for Apache Ambari during cluster creation.

The integration of Azure HDInsight and Azure Firewall, in order to restrict outbound traffic from the cluster to unauthorized destinations, is now generally available.

Azure HDInsight will deprecate G-series virtual machines (VMs) on October 31, 2019. After this date, you won't be able to create any new clusters using G-series VMs or to scale…

Azure CLI support for Azure HDInsight is now generally available. This enables administrators to easily manage their HDInsight clusters using simple Azure CLI commands.

Azure HDInsight now supports Apache Spark 2.4 and Apache Kafka 2.1. You can choose the Spark or the Kafka version that you want during cluster creation in the Azure portal.…

Azure HDInsight has updated against the vulnerabilities that affect the Linux kernel.

We are pleased to announce the integration of VSCode Python extension to handle PySpark interactive for HDInsight in VSCode. The Azure HDInsight Tools for VSCode leverages the Python extension to…

HDInsight for VSCode continues to boost the experience of Hive users with self-service exploratory capabilities. The HDInsight explorer in VSCode not only empowers you to browse Hive databases across…

We’ve corrected billing for Azure SQL Data Warehouse and other Azure services that impacted EA customers only.

HBase accelerated writes significantly improves the value proposition for low-latency, high- throughput NoSQL workloads.

Autoscale which automatically scales your Spark, Hive, or MapReduce HDInsight clusters up or down based on load or a pre-defined schedule, is now in preview.

We are introducing some important changes to support more fine-grained role-based access to sensitive configuration information from HDInsight clusters. As part of these changes, some user action may be required.

Four regions are available in China: China East, China North, China North 2 and China East 2.

New features in Hadoop 3 provide significant improvements to performance, stability, and availability, helping reduce total cost of ownership and accelerating time-to-value.

The networking resources (such as NICs and load balancers) for new Azure HDInsight clusters created in a virtual network are now provisioned in the same HDInsight cluster resource group.