Resolving the MSDeployPublish Error When Publishing Azure Web Jobs From Visual Studio 2015

I had a rather old Azure web job that I always deployed by creating a .zip myself and uploading it to the portal manually. I decided to use the nice feature inside Visual Studio 2015 that publishes the project for you. But when I walked through the wizard and hit “publish” I received the following error: Error MSB4057: The target "MSDeployPublish" does not exist in the project. I [Read More]

Simple Unit Tests For HDInsight C# SDK

I have been working on a project using the .NET SDK for Hadoop.  I wanted to add some unit tests to the project, so I ended up writing some fakes for HDInsightClient, JobSubmissionClientFactory and JobSubmissionClient. I was hoping I might be able to reuse some fakes from the SDK git repo, but it seems like their unit tests actually stand up an instance of Hadoop. I didn’t want [Read More]

Prototyping With Hive Using HDInsight

I’ve been doing a lot of prototyping with Hive lately and I really wanted to use an emulator to do the work.  For awhile, I was trying to do the prototyping using Powershell but kept getting NotSupported exceptions.  I finally decided to simply program against Hive itself which has turned out to work just great. I’m using Visual Studio to write my HQL as .sql files [Read More]

Connecting To The Azure Storage Emulator From The HDInsight Emulator

I was following the Getting Started instructions on using the HDInsight emulator and got stuck trying to connect to the Azure Storage Emulator:   hadoop fs -ls wasb://temp@storageemulator ls: `wasb://temp@storageemulator': No such file or directory Turns out that you must have a trailing slash, like this: hadoop fs -ls wasb://temp@storageemulator/ Maybe that’ll help someone out there… [Read More]