Building a Spark Application for HDInsight using IntelliJ Part 1 of 2

For developers with a Microsoft .NET background who want to get familiar with building Spark applications with Scala programming language, this blog post series is a walk through from installing the development tools and building a simple Spark application, then submit against an HDInsight Spark cluster. My HDInsight configuration is Spark 2.0 (HDI 3.5) with …

Continue reading Building a Spark Application for HDInsight using IntelliJ Part 1 of 2

Build Power BI Reports with HDInsight Spark Connector

With Power BI Desktop you can use the Azure HDInsight Spark BI Connector to get data from the Spark cluster to build reports. I have an HDInsight Spark 2.0 cluster with Azure Data Lake Store as the primary storage. Open Power BI Desktop Click Get Data Enter url of your HDInsight Spark cluster. Enter the …

Continue reading Build Power BI Reports with HDInsight Spark Connector

Text Analytics of Movie Reviews using Azure Data Lake, Cognitive Services and Power BI (part 2 of 2)

Part 1 of 2: Text Analytics of Movie Reviews using Azure Data Lake, Cognitive Services and Power BI (part 1 of 2) Take a csv file, analyze with an U-SQL script in Azure Data Lake Part 2 of 2:  Applicable Business Scenario Marketing or data analysts who need to review sentiments and key phrases of a […]

Text Analytics of Movie Reviews using Azure Data Lake, Cognitive Services and Power BI (part 1 of 2)

Part 1 of 2: Text Analytics of Movie Reviews using Azure Data Lake, Cognitive Services and Power BI (part 1 of 2) Take a csv file, analyze with an U-SQL script in Azure Data Lake Part 2 of 2: Text Analytics of Movie Reviews using Azure Data Lake, Cognitive Services and Power BI (part 2 of 2) …

Continue reading Text Analytics of Movie Reviews using Azure Data Lake, Cognitive Services and Power BI (part 1 of 2)

Azure Search: Pushing Content to an Index with the .NET SDK.

Blog Series Azure Search Overview Pushing Content To An Index with the .NET SDK I hold the opinion that for a robust indexing strategy, you would likely end up writing a custom batch application between your desired data sources and your defined Azure Search index. The pull method currently only supports data sources that reside …

Continue reading Azure Search: Pushing Content to an Index with the .NET SDK.

The Effects of Dropping Internal and External Hive Tables in HDInsight and ADLS

In my blog post Populating Data into Hive Tables in HDInsight, I have demonstrated populating an internal and an external hive table in HDInsight. The primary storage is configured with Azure Data Lake Store. To see the differences, I will demonstrate dropping both types of tables and observe the effects. This for the beginner audience. To recap …

Continue reading The Effects of Dropping Internal and External Hive Tables in HDInsight and ADLS

Azure Data Lake Analytics: Job Execution Time and Cost

Blog Series: Creating Azure Data Lake PowerShell and Options to upload data to Azure Data Lake Store Using Azure Data Lake Store .NET SDK to Upload Files Creating Azure Data Analytics Azure Data Lake Analytics: Database and Tables Azure Data Lake Analytics: Populating & Querying Tables Azure Data Lake Analytics: How To Extract JSON Files …

Continue reading Azure Data Lake Analytics: Job Execution Time and Cost