Posts Tagged ‘azure’

I recently started in the Fishers Youth Mentoring Initiative, and my mentee is a young man in junior high who really likes lizards. He showed me photos of them on his iPad, photos of his pet lizard, and informed me of many lizard facts. He’s also a talented sketch artist – showcasing many drawings of Pokemon, lizards and more. Oh, yeah, he’s also into computers and loves his iPad.

Part of the mentoring program is to help with school, being there as they adjust to growing up, and both respecting and encouraging their interests.

It just so happens that he had a science project coming up. He wasn’t sure what to write about. His pet lizard recently had an attitude shift, and he figured it was because it wasn’t getting as much food week over week. Changing that, he realized its attitude changed. So, he wanted to cover that somehow.

Seeing his interest in lizards, drawing, and computers I asked if we could combine them. I suggested we build an app, a “Reptile Tracker,” that would help us track reptiles, teach others about them, and show them drawings he did. He loved the idea.


We only get to meet for 30 minutes each week. So, I gave him some homework. Next time we meet, “show me what the app would look like.” He gleefully agreed.

One week later, he proudly showed me his vision for the app:

Reptile Tracker

I said “Very cool.” I’m now convinced “he’s in” on the project, and taking it seriously.

I was also surprised to learn that my expectations of “show me what it would look like” were different from what I received from someone both much younger than I and with a different world view. To him, software may simply be visualized as an icon. In my world, it’s mockups and napkin sketches. It definitely made me think about others’ perceptions!

True to software engineer and sort-of project manager form, I explained our next step was to figure out what the app would do. So, here’s our plan:

  1. Identify if there are reptiles in the photo.
  2. Tell them if it’s safe to pick it up, if it’s venomous, and so forth.
  3. Get one point for every reptile found. We’ll only support Lizards, Snakes, and Turtles in the first version.

Alright, time for the next assignment. My homework was to figure out how to do it. His homework was to draw up the Lizard, Snake, and Turtle that will be shown in the app.

Challenge accepted!

I quickly determined a couple key design and development points:

  • The icon he drew is great, but looks like a drawing on the screen. I think I’ll need to ask him to draw them on my Surface Book, so they have the right look. Looks like an opportunity for him to try Fresh Paint on my Surface Book.
  • Azure Cognitive Services, specifically their Computer Vision solution (API), will work for this task. I found a great article on the Xamarin blog by Mike James. I had to update it a bit for this article, as the calls and packages are a bit different two years later, but it definitely pointed me in the right direction.

Writing the Code

The weekend came, and I finally had time. I had been thinking about the app the remainder of the week. I woke up early Saturday and drew up a sketch of the tracking page, then went back to sleep. Later, when it was time to start the day, I headed over to Starbucks…


I broke out my shiny new MacBook Pro and spun up Visual Studio Mac. Xamarin Forms was the perfect candidate for this project – cross platform, baby! I started a new Tabbed Page project, brought over some code for taking photos with the Xam.Plugin.Media plugin and resizing them, and the beta Xamarin.Essentials plugin for eventual geolocation and settings support. Hey, it’s only the first week Smile

Side Note: Normally I would use my Surface Book. This was a chance for me to seriously play with MFractor for the first time. Yay, even more learning this weekend!

Now that I had the basics in there, I created the interface for the Image Recognition Service. I wanted to be able to swap it out later if Azure didn’t cut it, so Dependency Service to the rescue! Here’s the interface:

using System.IO;
using System.Threading.Tasks;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
namespace ReptileTracker.Services
     public interface IImageRecognitionService
         string ApiKey { get; set; }
         Task<ImageAnalysis> AnalyzeImage(Stream imageStream);

Now it was time to check out Mike’s article. It made sense, and was close to what I wanted. However, the packages he referenced were for Microsoft’s Project Oxford. In 2018, those capabilities have been rolled into Azure as Azure Cognitive Services. Once I found the updated NuGet package – Microsoft.Azure.CognitiveServices.Vision.ComputerVision – and made some code tweaks, I ended up with working code.

A few developer notes for those playing with Azure Cognitive Services:

  • Hold on to that API key, you’ll need it
  • Pay close attention to the Endpoint on the Overview page – you must provide it, otherwise you’ll get a 403 Forbidden


And here’s the implementation. Note the implementation must have a parameter-less constructor, otherwise Dependency Service won’t resolve it.

using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Threading.Tasks;
using ReptileTracker.Services;
using Xamarin.Forms;
[assembly: Dependency(typeof(ImageRecognitionService))]
namespace ReptileTracker.Services
    public class ImageRecognitionService : IImageRecognitionService
        /// <summary>
        /// The Azure Cognitive Services Computer Vision API key.
        /// </summary>
        public string ApiKey { get; set; }
        /// <summary>
        /// Parameterless constructor so Dependency Service can create an instance.
        /// </summary>
        public ImageRecognitionService()
        /// <summary>
        /// Initializes a new instance of the <see cref="T:ReptileTracker.Services.ImageRecognitionService"/> class.
        /// </summary>
        /// <param name="apiKey">API key.</param>
        public ImageRecognitionService(string apiKey)
            ApiKey = apiKey;
        /// <summary>
        /// Analyzes the image.
        /// </summary>
        /// <returns>The image.</returns>
        /// <param name="imageStream">Image stream.</param>
        public async Task<ImageAnalysis> AnalyzeImage(Stream imageStream)
            const string funcName = nameof(AnalyzeImage);
            if (string.IsNullOrWhiteSpace(ApiKey))
                throw new ArgumentException("API Key must be provided.");
            var features = new List<VisualFeatureTypes> {
            var credentials = new ApiKeyServiceClientCredentials(ApiKey);
            var handler = new System.Net.Http.DelegatingHandler[] { };
            using (var visionClient = new ComputerVisionClient(credentials, handler))
                    imageStream.Position = 0;
                    visionClient.Endpoint = "";
                    var result = await visionClient.AnalyzeImageInStreamAsync(imageStream, features);
                    return result;
                catch (Exception ex)
                    Debug.WriteLine($"{funcName}: {ex.GetBaseException().Message}");
                    return null;

And here’s how I referenced it from my content page:

pleaseWait.IsVisible = true;
pleaseWait.IsRunning = true;
var imageRecognizer = DependencyService.Get<IImageRecognitionService>();
imageRecognizer.ApiKey = AppSettings.ApiKey_Azure_ImageRecognitionService;
var details = await imageRecognizer.AnalyzeImage(new MemoryStream(ReptilePhotoBytes));
pleaseWait.IsRunning = false;
pleaseWait.IsVisible = false;

var tagsReturned = details?.Tags != null 
                   && details?.Description?.Captions != null 
                   && details.Tags.Any() 
                   && details.Description.Captions.Any();

lblTags.IsVisible = true; 
lblDescription.IsVisible = true; 

// Determine if reptiles were found. 
var reptilesToDetect = AppResources.DetectionTags.Split(','); 
var reptilesFound = details.Tags.Any(t => reptilesToDetect.Contains(t.Name.ToLower()));  

// Show animations and graphics to make things look cool, even though we already have plenty of info. 
await RotateImageAndShowSuccess(reptilesFound, "lizard", details, imgLizard);
await RotateImageAndShowSuccess(reptilesFound, "turtle", details, imgTurtle);
await RotateImageAndShowSuccess(reptilesFound, "snake", details, imgSnake);
await RotateImageAndShowSuccess(reptilesFound, "question", details, imgQuestion);

That worked like a champ, with a few gotchas:

  • I would receive a 400 Bad Request if I sent an image that was too large. 1024 x 768 worked, but 2000 x 2000 didn’t. The documentation says the image must be less than 4MB, and at least 50×50.
  • That API endpoint must be initialized. Examples don’t always make this clear. There’s no constructor that takes an endpoint address, so it’s easy to miss.
  • It can take a moment for recognition to occur. Make sure you’re using async/await so you don’t block the UI Thread!

Prettying It Up

Before I get into the results, I wanted to point out I spent significant time prettying things up. I added animations, different font sizes, better icons from The Noun Project, and more. While the image recognizer only took about an hour, the UX took a lot more. Funny how that works.

Mixed Results

So I was getting results. I added a few labels to my view to see what was coming back. Some of them were funny, others were accurate. The tags were expected, but the captions were fascinating. The captions describe the scene as the Computer Vision API sees it. I spent most of the day taking photos and seeing what was returned. Some examples:

  • My barista, Matt, was “a smiling woman working in a store”
  • My mom was “a smiling man” – she was not amused

Most of the time, as long as the subjects were clear, the scene recognition was correct:


Or close to correct, in this shot with a turtle at Petsmart:


Sometimes, though, nothing useful would be returned:


I would have thought it would have found “White Castle”. I wonder if it won’t show brand names for some reason? They do have an OCR endpoint, so maybe that would be useful in another use case.

Sometimes, even though I thought an image would “obviously” be recognized, it wasn’t:


I’ll need to read more about how to improve accuracy, if and whether that’s even an option.

Good thing I implemented it with an interface! I could try Google’s computer vision services next.

Next Steps

We’re not done with the app yet – this week, we will discuss how to handle the scoring. I’ll post updates as we work on it. Here’s a link to the iOS beta.

Some things I’d like to try:

  • Highlight the tags in the image, by drawing over the image. I’d make this a toggle.
  • Clean up the UI to toggle “developer details”. It’s cool to show those now, but it doesn’t necessarily help the target user. I’ll ask my mentee what he thinks.

Please let me know if you have any questions by leaving a comment!

Want to learn more about Xamarin? I suggest Microsoft’s totally awesome Xamarin University. All the classes you need to get started are free.

Update 2018-11-06:

  • The tags are in two different locations – Tags and Description.Tags. Two different sets of tags are in there, so I’m now combining those lists and getting better results.
  • I found I could get color details. I’ve updated the accent color surrounding the photo. Just a nice design touch.

I recently deployed an Azure Cloud Service with Remote Desktop enabled. However, when I went to connect to it on port 3389, the server refused the connection. I remember there was something I had to do, but I had never written down the steps. So, here’s what you need to do, in case you’re looking 🙂

Note: I use Remote Desktop Connection Manager, a.k.a. RDCMan, also from Microsoft, instead of the standard RDC client. I feel it’s much better, more configurable, and great if you need to work with many remote desktops. I’ve love to know why they don’t include it in Windows!

Step 1: Find your Cloud Service and Slot in Azure Portal

In Azure, find your Cloud Service. Also select the slot to which you want to connect, such as Production or Staging.

Step 2: Select the Roles and Instances option

You’ll see it on the left.

Step 3: Choose the item to which you want to connect and click Connect

For example, your web role instance. This will download an .RDP file. You can double-click this file to connect. Ooh! Neat!

Step 4: If you’re using RDCMan…

To connect with RDCMan, you’ll need to grab the Cookie: something something something string out of the RDP file. Open it in Notepad++, or your text editor of choice, and grab that value. Ignore the s: text.

In RDCMan, for the VM, add the string under Connection Settings tab in the Load balance config textbox.

Step 5: You’re Connected!




Have you been wondering how to access the Azure Multi-Factor Authentication Settings in the Azure Classic Portal without first having to create an Azure account? I figured this out a few days ago, having an Office 365 tenant, and wanting to use the EMS and Azure Active Directory Premium features. Following Microsoft’s instructions, it said to go to the Azure Classic Portal. The problem is, Office 365 doesn’t include an Azure subscription, it just includes Azure Active Directory, which you manage through the “modern” Azure portal. Unfortunately, the Trusted IPs and MFA capabilities are managed through the Azure Classic Portal, which you can’t directly access without an Azure subscription.

So, here’s what you do:

  1. Go to
  2. Click Admin to open the admin tools
  3. In the search box, type MFA
  4. Select the Multi-factor authentication search result
  5. Click the link to open the Manage multi-factor authentication link
  6. There you go – manage MFA in your Azure AD to your heart’s content!


I recently ran into a need to use the LAME MP3 encoder in a customer’s website. Problem was, once I deployed to Azure, I received an error of “Unable to load DLL libmp3lame.32.dll”. Uh oh! “But it’s in the bin folder!” I screamed silently at Starbucks. So, I binged the issue and found a good answer on StackOverflow. I’m sharing here because it helped unstick me, and I imagine others may be running to this issue with libraries other than LAME.

I ended up adding the function to my Global.asax, in addition to importing namespaces System.IO and System.Linq:

/// <summary>
/// Updates PATH variable in hosting instance to allow referring to items in this project's /bin folder.
/// Very helpful with Azure.
/// </summary>
public static void CheckAddBinPath()
    // find path to 'bin' folder
    var binPath = Path.Combine(new string[] { AppDomain.CurrentDomain.BaseDirectory, "bin" });
    // get current search path from environment
    var path = Environment.GetEnvironmentVariable("PATH") ?? "";
    // add 'bin' folder to search path if not already present
    if (!path.Split(Path.PathSeparator).Contains(binPath, StringComparer.CurrentCultureIgnoreCase))
        path = string.Join(Path.PathSeparator.ToString(), new string[] { path, binPath });
        Environment.SetEnvironmentVariable("PATH", path);

Then in Application start I simply added:

// Sometimes files aren't loaded properly from bin. Hint to the app to load from /bin, too.

I hope that helps!

I struggled with this for a few days while trying to convert a Silverlight video player to HTML5, and finally found an answer. Posting here in case anyone else is having trouble!

You need to specify the format as MPEG DASH to get it to smoothstream the MP4 file to the HTML5 video player. This is done by adding a format parameter to the manifest URL, as follows:
Note the (format=mpd-time-csf) at the end of the URL. There are a number of other formats you can stream, including the Silverlight SmoothStream, Adobe’s streaming format, Apple’s HTTP Live Streaming for iOS devices, and more. This is all done for you automatically by Azure’s Media Services. Pretty darn cool.
I struggled to find this, too, so quite happy I finally got things working. Here’s the source URL from Microsoft for more details:

I’m relatively new to Azure deployments, but the more I use them, the more I like the service. Unfortunately, it’s not WYSIWYG with deployments. What you see on IIS Express when development is not always what you’ll get after an Azure deployment. One issue I’ve come across is the MIME mappings aren’t the same, or don’t exist at all, and that’s preventing various file types, such as SVG images and WOFF2 fonts from being served. I also noticed that fixing Azure’s MIME mapping busted my AngularJS support in IIS Express – whoops!

Using the magic of web.config transforms, we can fix this for our release deployments. If you expand your web.config file, you’ll see web.Debug.config and web.Release.config. These files enable you to insert, replace, and remove settings based on your build configuration. Obviously, if you had multiple build configs, such as for different hosting environments, you’d insert those config names in addition web.*.config files.

To add SVG support, we need to insert our additional MIME types into the web.config. There’s no reason to do this in the master web.config, because it’s only necessary during release. This same tactic works very well for swapping the SMTP mail mailSettings section based on the hosting environment’s needs. For example, I swap localhost, where I use PaperCut to monitor sent email, to the actual settings upon deployment.

Below, you’ll see the fully modified web.Release.config from a recent deployment. This one worked perfectly for adding SVG and some missing font file extension support. You’ll notice I’m adding a new section under <system.webServer>. Note that I do not mark the webServer tag with an xdt:Transform. I don’t want to replace the entire section. I simply need to add the staticContent section to override some of the settings already configured in Azure. There are other options for xdt:Transform, such as Remove and Replace. This is a very powerful feature and I encourage you to learn more about it from Microsoft.

    <staticContent xdt:Transform="Insert">

I hope this helps!


<?xml version="1.0"?>
<!-- For more information on using Web.config transformation visit -->
<configuration xmlns:xdt="">
    <!-- Add support for video files and other non-standard file request types. This breaks AngularJS support in IISExpress, hence why it's here instead. -->
    <staticContent xdt:Transform="Insert">
      <!-- if you don't remove certain extensions first, the site won't load, whoops! -->
      <remove fileExtension=".svg" />
      <remove fileExtension=".svgz" />
      <remove fileExtension=".eot" />
      <remove fileExtension=".ttf" />
      <remove fileExtension=".woff" />
      <remove fileExtension=".woff2" />
      <mimeMap fileExtension=".mp4" mimeType="video/mp4" />
      <mimeMap fileExtension=".ogv" mimeType="video/ogg" />
      <mimeMap fileExtension=".webm" mimeType="video/webm" />
      <mimeMap fileExtension=".svg" mimeType="image/svg+xml"/>
      <mimeMap fileExtension=".svgz" mimeType="image/svg+xml"/>
      <mimeMap fileExtension=".eot" mimeType="application/" />
      <mimeMap fileExtension=".ttf" mimeType="application/octet-stream" />
      <mimeMap fileExtension=".woff" mimeType="application/font-woff" />
      <mimeMap fileExtension=".woff2" mimeType="application/font-woff2" />