I recently ran into this issue. One of our iPads, which had worked with TestFlight apps before, was now giving an error: “TestFlight is currently unavailable.”
Suffice it to say, this was a blocker to testing our customer’s app.
Looking at Apple’s support forums and StackOverflow, this is a common issue, and I saw no posts with solutions. Just people saying “this happens from time to time” or simply giving up because it worked on another device with another account.
So how do you solve it? A coworker, Jeff, had a suggestion – had we added the user to our Apple account?
I hadn’t thought about this. Normally, you just invite people via TestFlight in App Store Connect and it just works. But the difference here was the iPad was a corporate managed device, via Intune, and the user was a managed user, using single sign-on, aka SSO. It wasn’t a normal personal Apple account.
Click the + to add the Apple account to the list of users. Set their access level to Developer. (I didn’t try this WITHOUT that, feel free to try and update me on success)
Make sure they accept the invite, otherwise this won’t work. Apple’s invites can sometimes be quarantined/flagged as spam, so be on the lookout.
Now go to TestFlight and invite that same account to the app and ensure they have access to the necessary builds. I’m assuming you already know how to do this part, so I’m leaving those screenshots out.
Problem solved. “TestFlight Unavailable” error went away.
The “Users and Access” option.The “New User” dialog, with Developer option checked.
Other Solutions to Try
Has the app build expired?
Are there pending Apple agreements that need to be signed?
Has a certificate expired?
Did you invite the user to the right tester list?
Are Apple’s servers down? Maybe wait a day…
Do you have Internet access?
If you can’t install TestFlight (different problem, really), and it’s a managed device, have you talked with IT to have it added as an allowed / auto-installed app?
Finally… my bourbon, whiskey, tequila, mezcal, wine, whatever you want to drink + taste + collect app… Liq… is now available for both mobile platforms. What an adventure!
Originally, Liq was a website – https://liq.live. It was the easiest place to start, and start figuring out what people want. Plus, any updates I push would be immediately available to everyone. No waiting on app updates, supporting older versions, writing an API and more advanced caching infrastructure… And no app store reviews to get through, which can be troublesome in some cases.
But my users have spoken! “We want an app!” So I got to work a couple months ago and started building. Thankfully, most of the code was reusable. I found out quickly, the mobile app was the way to go. I could guarantee the experience. I could code for phones vs. tablets. And I could make it insanely simple to have such a useful tasting + collection tool anywhere users went.
Once it was done and tested – thanks Club Marzipan! – I was ready to submit. I’ve done this countless times. Every time, you cross your fingers… Let the story begin.
The iOS Story
Usually, Apple App Review is the tough part. Who knows what unknown rule you might break when submitting your first app release. Once you’re past that, the untrusting eyes are mostly off you. But still… you have to get over that first hurdle.
My initial submission was rejected. Apple found a bug I had never tested for, and they rightfully rejected the app. But not only did they reject it for the bug, they rejected it saying it’s nothing better than a website and I should add native features.
Well, that wasn’t fair, but I could see their point. Because of the bug, you couldn’t use the app. So, you wouldn’t see any of the features. I thanked them for reporting the bug, and that I disagreed about the “native” part. When working properly, Liq uses many native features. Native camera. Native photo access. Native sharing. Plus it’s, well, a native compiled app. Basically, every native feature Facebook uses except Location and Video, neither of which I need. Apple wrote back it doesn’t matter, but I ignored that – I told them “you’ll see in my next upload.”
I fixed the bug – again, thanks Apple, that was a good catch! – and re-uploaded. They came back about a day later with a new rejection reason. The bug was resolved, so that issue was removed, as was the “native” issue – phew! – that would have been an unnecessary rewrite. This time they pointed out I had no way to request account deletion. Fair point. I got to work.
I had the accout deletion fixed within a few hours. I tested the heck out of it. It worked great. I resubmitted and hoped for the best.
Two days later is was approved. w00t!
I did have some additional fixes. I let the original version get out there – it was only going to be out for a day – while I fixed some more bugs and tightened things up. Apple Reviews go much faster when you already a published verison of the app. I submitted the update and it was reviewed and approved the same day. Boom, proof.
Now, on to Android…
The Android Story
Usually, Google Play – the Android store – submission is a slam dunk. Google appears to use AI and another automated code-analysis system. I recall it being named something like The Bouncer. I don’t recall the exact name – it’s been a long time and I imagine things have changed. Anyway, given Apple’s issues were largely just Apple requirements, I figured I’d be reviewed within a day. You never know though!
I started to worry when the review had taken more than 2 days. A status of In Review for more than a few hours is uncommon. I was concerned. Alas, the rejection email came in. And it wasn’t what I expected:
First, I don’t know what “all parts of the metadata” means. I think it’s telling me my provided app screenshots and what’s in the app don’t match its description. Specifically, “track your bottles” and “see what you’ve tasted.” Let’s look at their screenshot:
So that’s their “proof” my app is misleading. But what’s this? Their screenshots show exactly what my app description details. Tastings and Collection. It’s all right there.
I’m guessing this is just a case of AI making a mistake judging my app. I don’t blame Google – they probably get a lot of submissions, and have minimal resources to look at them. Remember all those layoffs?
I submitted an appeal. They said it would take 2 days.
I received an email saying it might take longer than 2 days. OK, let’s be patient.
One week later, I finally received a notice that my appeal was accepted. All I had to do – though it wasn’t entirely clear in the email – was resubmit my app. I did that and started to wait.
Two days later, it still was In Review. “Oh, no…” I was thinking to myself… Not this again… Don’t reject me again, AI.
Thankfully, a few days after, my app was finally approved. Wow, unexpected. But I’m happy 😀🎉
While in Nashville, having just torched my insides with a fiery hot chicken sandwich from Prince’s, I passed by the all-glass enclosed Apple store on the corner of 5th & Broadway. It was about 6pm, and observations would indicate the evening had already started in the morning for most people. It was my last night in town and the Apple store got me thinking about the Vision Pro, my Quest 3, the 250,000 units Apple had sold to date, and of the articles claiming you could get a demo. So I walked in and asked “Can I get a demo of the Vision Pro?” “Sure! We may have an appointment left,” the friendly associate exclaimed. 7pm was available. I handed them my deets, and went off to drop some shopping goodies off at my hotel a few blocks away.
The Pre-Demo Setup
Before I even left the store, Apple had texted me a pre-demo survey. It was primarily concerned with whether I wore glasses (I do now), and whether I used them for near or far sightedness, or both. I answered the questions and Apple told me I was all set.
Apple’s pre-demo survey recommended I use the Zeiss inserts.
I strolled back into the store close to 7pm. They greeted me quickly with “You’re back!” Well, “I’m a man of my word” I exhorted, and followed them to the back of the store for the demo. Sam would be talking me through the demo. But first they needed my glasses so they could measure them with a fancy, expensive looking measurement machine. I think I’ve seen the same device at Warby Parker. They needed this to set up the Vision Pro for my exact prescription. This begged the question, “So what if my prescription changes? Do I need a new set of inserts?” “Yes, you would need to order new inserts” she explained. I didn’t see her add any inserts, so I was a bit confused by this, but why die on that hill? My guess is the Vision Pro adjusts by itself once provided the script details, but who knows. I sure hope I wouldn’t need to buy new adapters, err “inserts,” after already spending thousands of dollars on this thing. And of course – at this time – you can only buy Apple’s special Zeiss inserts, which I’m sure are a pretty penny.
The lens scanning machine.
After my eyes were ready, I also had to use an iPhone to scan my face. This process wasn’t working well until I moved to the solid-colored wall. The app just kept missing the scan. Sam was a bit frustrated as well, but she kept her cool.
Now, keep in mind, I’ve owned a Meta Quest 3 for a few months now. I was 100% comparing the setup process of that under $500 device to the setup process of a $3,500 (base!!) device here. With the Quest, I just put the unit on my head after some simple setup, and just kept my glasses on. I’m curious how much of this pomp and circumstance is actually necessary, or might be removed in a future software update for the Vision Pro.
Seeing all the work and equipment that went into just getting the unit to be “ready for me” helped me understand the price point. The optical equipment, the personnel, the technology for such a customized experience, has to come from somewhere. Given Apple would have raked in around $1.125B after selling 250K units, I hope they’ve recouped their costs. Now if only there were an iFixit teardown… Oh wait, there is!😀
Note that the entire demo was done while in a sitting position. I was sitting on a chair next to a wooden table. One other person was experiencing a demo at the same time.
Fit and Finish
Sam showed me what the buttons do – a button for the see-through mode, and a “digital crown” like the Apple watch. She also showed me the exact way to place the Vision Pro on my head. Thumb under the nosepiece, and four fingers on the top. Don’t touch the front! I asked what would happen if I did – would it just look ugly? She said yeah, it wouldn’t look good, but otherwise probably nothing. I followed her advice and put the unit on my head. I used the right-hand dial to tighen the unit as close as possible to my liking. Note that, because of the Zeiss inserts, I did not need my glasses on for the demo.
The “eyes” passthrough wasn’t part of the demo.
Once the Vision Pro was on my noggin’ I realized how heavy it is. I have a Quest 3 at home. This unit clearly felt heavier. It wasn’t uncomfortable, but it did feel like I had a decent size computer on my head, which of course I did. Sam suggested I move the strap around a bit. After somem finagling, I figured this was as good as it was going to get. It didn’t feel like it was going to fall off. It just felt front-heavy, like the top-heavy feeling I get when my bourbon-belly body is on rollerblades. I did a search, and the Quest 3 is around 515 grams, while the Vision Pro is around 532.
Moving my fingers around I also found the digital crown to be too small. I would use this control device later, and I have to tell you, it needs to be bigger. When you can’t see something, and you want to do small movements with it, and it’s already small, it’s frustrating. Yes, it’s cool, and it fits with the Apple ecosystem, but this needed to be adjusted.
The Digital Crown.
Now, the quality of the materials is top-notch. The strap was incredibly comfortable and disappeared as I used the product. Everything looks clean and precisely engineered. Even the carrying case looks like a high-end The North Face affair. The heaviness did not disappear, however.
The Demo
I should have mentioned earlier that Sam explained she could see everything I was seeing. She had an iPad wirelessly streaming the feed from the Vision Pro. She also had an iPhone that appeared to have the demo script. It was clear Apple wants this demo staged and not free-form. When I would pinch and zoom or move a window before being prompted, Sam would gently verbally nudge me with “Please wait for me.” Sorry, Sam!
First things first was the setup mode. The Vision Pro walked me through mapping my touching and vision. The vision part was interesting – I had to look at each dot on the screen and then pinch my fingers together to “tap” it. Moving my eyes, not necessarily my head, would move an invisible pointer. At the center of my vision – what I’m looking at – becomes what’s selected. It was also incredibly clear and vibrant – so whatever the Zeiss and vision calibration did, it did it well.
The experience is also fascinating from a UI and UX perspective. The center stays focused while items in my “peripheral” vision go out of focus when I move my head, coming into slightly better focus when I stop. In practice, this worked very well. However, the selecting and tapping part was not 100%. I’d say 3 out of 10 times – 30% – when I tried tapping something, the Vision Pro wouldn’t register the tap. Perhaps my hand was under the demo table and I didn’t realize it – but moving my hand closer to the device or further in front of me seemed to solve the issue. I also had to ensure I didn’t lose focus on what I wanted to tap, or I would “miss” or tap the wrong item. After some time using this, I’m sure it would become natural. For the most part it was – but it was clear after 30 minutes Apple has some tweaking to do in its UX, and I can see why this is a scripted demo. But still, damn, it’s amazing.
Once setup was complete, the Apple logo appeared, and I was greeted with the Home Screen. Yes, it looks like the typical iOS home screen layout, just in front of you with your surroundings in semi-transparent fashion in the background. You can tune out your surroundings by rotating the digital crown. I was only allowed to use on of the virtual backgrounds. Sam wouldn’t let me play with others, and she could clearly see via her iPad if I broke the rules. What I did experience, though, was a calming lakeside landscape. It even started raining while I was “there” and that was quite cool, and would have been calming had I not been in the middle of an Apple store. The speakers were loud enough for me to hear the raindrops, but I wasn’t there for that experience. Before you ask – no, I didn’t get a chance to set up the see-through mode that shows my eyes. That’s not part of the demo.
There are three basic gestures on the Vision Pro: Tap, Tap and Drag, and Pinch/Pull to Zoom. The first two are single-hand gestures performed with your dominant hand. The latter requires both hands, and gives you the feeling you’re in [insert Sci-Fi movie here] and manipulating virtual screens in the clear space in front of you. Yeah, it’s pretty cool. Another verbal wrist slap from Sam for me getting ahead of the game.
Demo 1 – Photos
The first demo was launching the classic Apple Photos app. There were many to choose from. Some were “flat” while others had the 3D depth old 3D Android phones were capable of many years ago. Remember the HTC Evo 3D? The flat photo was, well, a photo, and I could zoom in and out as expected. It was perfectly clear, and the colors were sharp and realistic. The 3D photo had true depth, and was shot on an iPhone 15 Pro. Both the 15 Pro and Pro Max support creating 3D and immersive content for the Vision Pro. Apple’s pushing those devices as content creation catalysts, understandbly. Because it was a scripted store demo, I didn’t ask for additional details like format support and technical details. My understanding is other 3D formats are supported, so you’re not limited to just Apple ecosystem solutions.
Demo 2 – Videos
Now for the fun part – video. There was no demo of a flat video here, and that’s fine. Who cares? Every headset does it. You’re not spending $3,500+ for a simple movie theater. There were two demos – one 3D video that wasn’t immersive, meaning it didn’t surround you, and another immersive sports video. The 3D video was cool – a family blowing out the candles on a cake. The frame rate seemed low, maybe 30fps, and reminded me of 3D video from those old 3D Android phones I talked about. It was neat that it was “large” in front of me, but it wasn’t mind-blowing due to having seen it before. Now, I’d like to know if the Quest 3 can do the same. Sam did not appreciate that I played the video more than once. To be fair, she had a lot of patience with me – thank you Sam!
The real treat was the immersive video compilation. It had many immersive videos, all being narrated by someone telling me how great “living the action” is. One was shot with a 360 degree (I think) camera placed on a soccer goal and I could see the game and the ball being kicked into the net. Another was a mountain landscape and I was watching the climber. Another was shot behind first base during a double play. You get the point – incredible action sequences to make you feel like you are there. And it did. It was exhilarating. I recall Sam explaining it was all 8K video. I asked if the screens themselves were 8K, but she wasn’t sure. The detail was phenomenal. Absolutely stunning.
Is there a new market here?
My first thought was Apple TV Plus – what if they started offering this type of content? Is that where it’s headed? I don’t know if it’s viable. Many of you may remember the many, many, MANY failures of trying to bring 3D into the home. Projectors, TVs, special glasses – and the fact 30% of human beings can’t watch 3D content without getting nauseous – it never worked. But they also didn’t have the content, other than more expensive 3D versions of Blu-ray discs. Could Apple stream this type of content? Could they convince people to wear these headsets while watching events such as concerts? I’m not convinced about sports, as I can’t see a bunch of people wearing headsets and drinking beer… Now that I’d like to see. If people generally look funny in VR, that would be a hoot. My point is, Apple certainly has the market position and technologies to make something happen here. What, I’m not yet sure… And Meta may be willing to play ball. If the monopoly regulators have their way, it may be a perfect match…
Demo 3 – Web Browsing and Compatible Apps
The last demo was showing that I could browse the web (Safari, yay?) and run “compatible apps” from the App Store. Meh. It’s iOS, so no surprises here. Cool, but no compelling killer app. The demo app Sam wanted me to run was a cooking app. I won’t be wearing a $3,500 + tax headset near the stove.
The Missing Demo
The Vision Pro content demos were impressive, to be sure. But where was the killer app to sell me on this $3,500 device? Sam kept telling me how this was a “Spatial Computing” device. But never did I see an example of spatial computing. I saw spatial consuming but not spatial creating. I would love to see the results of a survey of the 250,000 purchasers of this product explaining why… and what their income bracket is.
Final Thoughts
I took the Vision Pro off my head and handed it back to Sam. I did this the proper way… thumb under the nosepiece and four fingers on top. I thanked her for the experience and agreed it was quite impressive. I asked how many of these they sold each day. She couldn’t say, other than some people come in and simply buy one outright, no demo needed. It wouldn’t have been fair to ask her why – she’s just selling the unit, and knows fervent Apple fans with an Apple Credit Card are often willing to buy more Apple products (I jest).
But after the demo, I had no incentive to purchase the unit. There was nothing about it, at least during this entertaining 30 minute demo, that left a compelling reason on the table. Certainly not one that made me go “Gosh, I wish my Quest 3 did that!” I do need to determine if the Quest 3, at 1/6 the price, can do 3D video (UPDATE: It does!). But the Vision Pro demo was all about content consumption, and the Quest 3 does effectively the same thing in spades. Oh, and I can play VR games made specifically for its platform (noticeably absent from the Apple demo, but also understandable given the time constraints).
I also left with the feeling of possibility. What the Vision Pro represents, and what could come from such technology, finally, in the content consumption space. And maybe, eventually, in the content creation space, if Apple’s professional applications arm releases whatever they’ve got cooking. Who cares what you call it – spatial computing, VR, or otherwise – if you build something truly compelling.
Either way, the demo was worth it, I got my technologist buzz and my analyst gears working, and still have $3,500 to spend on something else.
I recently started in the Fishers Youth Mentoring Initiative, and my mentee is a young man in junior high who really likes lizards. He showed me photos of them on his iPad, photos of his pet lizard, and informed me of many lizard facts. He’s also a talented sketch artist – showcasing many drawings of Pokemon, lizards and more. Oh, yeah, he’s also into computers and loves his iPad.
Part of the mentoring program is to help with school, being there as they adjust to growing up, and both respecting and encouraging their interests.
It just so happens that he had a science project coming up. He wasn’t sure what to write about. His pet lizard recently had an attitude shift, and he figured it was because it wasn’t getting as much food week over week. Changing that, he realized its attitude changed. So, he wanted to cover that somehow.
Seeing his interest in lizards, drawing, and computers I asked if we could combine them. I suggested we build an app, a “Reptile Tracker,” that would help us track reptiles, teach others about them, and show them drawings he did. He loved the idea.
Planning
We only get to meet for 30 minutes each week. So, I gave him some homework. Next time we meet, “show me what the app would look like.” He gleefully agreed.
One week later, he proudly showed me his vision for the app:
I said “Very cool.” I’m now convinced “he’s in” on the project, and taking it seriously.
I was also surprised to learn that my expectations of “show me what it would look like” were different from what I received from someone both much younger than I and with a different world view. To him, software may simply be visualized as an icon. In my world, it’s mockups and napkin sketches. It definitely made me think about others’ perceptions!
True to software engineer and sort-of project manager form, I explained our next step was to figure out what the app would do. So, here’s our plan:
Identify if there are reptiles in the photo.
Tell them if it’s safe to pick it up, if it’s venomous, and so forth.
Get one point for every reptile found. We’ll only support Lizards, Snakes, and Turtles in the first version.
Alright, time for the next assignment. My homework was to figure out how to do it. His homework was to draw up the Lizard, Snake, and Turtle that will be shown in the app.
Challenge accepted!
I quickly determined a couple key design and development points:
The icon he drew is great, but looks like a drawing on the screen. I think I’ll need to ask him to draw them on my Surface Book, so they have the right look. Looks like an opportunity for him to try Fresh Paint on my Surface Book.
Azure Cognitive Services, specifically their Computer Vision solution (API), will work for this task. I found a great article on the Xamarin blog by Mike James. I had to update it a bit for this article, as the calls and packages are a bit different two years later, but it definitely pointed me in the right direction.
Writing the Code
The weekend came, and I finally had time. I had been thinking about the app the remainder of the week. I woke up early Saturday and drew up a sketch of the tracking page, then went back to sleep. Later, when it was time to start the day, I headed over to Starbucks…
I broke out my shiny new MacBook Pro and spun up Visual Studio Mac. Xamarin Forms was the perfect candidate for this project – cross platform, baby! I started a new Tabbed Page project, brought over some code for taking photos with the Xam.Plugin.Media plugin and resizing them, and the beta Xamarin.Essentials plugin for eventual geolocation and settings support. Hey, it’s only the first week
Side Note: Normally I would use my Surface Book. This was a chance for me to seriously play with MFractor for the first time. Yay, even more learning this weekend!
Now that I had the basics in there, I created the interface for the Image Recognition Service. I wanted to be able to swap it out later if Azure didn’t cut it, so Dependency Service to the rescue! Here’s the interface:
using System.IO;
using System.Threading.Tasks;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
namespace ReptileTracker.Services
{
public interface IImageRecognitionService
{
string ApiKey { get; set; }
Task<ImageAnalysis> AnalyzeImage(Stream imageStream);
}
}
Now it was time to check out Mike’s article. It made sense, and was close to what I wanted. However, the packages he referenced were for Microsoft’s Project Oxford. In 2018, those capabilities have been rolled into Azure as Azure Cognitive Services. Once I found the updated NuGet package – Microsoft.Azure.CognitiveServices.Vision.ComputerVision – and made some code tweaks, I ended up with working code.
A few developer notes for those playing with Azure Cognitive Services:
Hold on to that API key, you’ll need it
Pay close attention to the Endpoint on the Overview page – you must provide it, otherwise you’ll get a 403 Forbidden
And here’s the implementation. Note the implementation must have a parameter-less constructor, otherwise Dependency Service won’t resolve it.
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Threading.Tasks;
using ReptileTracker.Services;
using Xamarin.Forms;
[assembly: Dependency(typeof(ImageRecognitionService))]
namespace ReptileTracker.Services
{
public class ImageRecognitionService : IImageRecognitionService
{
/// <summary>
/// The Azure Cognitive Services Computer Vision API key.
/// </summary>
public string ApiKey { get; set; }
/// <summary>
/// Parameterless constructor so Dependency Service can create an instance.
/// </summary>
public ImageRecognitionService()
{
}
/// <summary>
/// Initializes a new instance of the <see cref="T:ReptileTracker.Services.ImageRecognitionService"/> class.
/// </summary>
/// <param name="apiKey">API key.</param>
public ImageRecognitionService(string apiKey)
{
ApiKey = apiKey;
}
/// <summary>
/// Analyzes the image.
/// </summary>
/// <returns>The image.</returns>
/// <param name="imageStream">Image stream.</param>
public async Task<ImageAnalysis> AnalyzeImage(Stream imageStream)
{
const string funcName = nameof(AnalyzeImage);
if (string.IsNullOrWhiteSpace(ApiKey))
{
throw new ArgumentException("API Key must be provided.");
}
var features = new List<VisualFeatureTypes> {
VisualFeatureTypes.Categories,
VisualFeatureTypes.Description,
VisualFeatureTypes.Faces,
VisualFeatureTypes.ImageType,
VisualFeatureTypes.Tags
};
var credentials = new ApiKeyServiceClientCredentials(ApiKey);
var handler = new System.Net.Http.DelegatingHandler[] { };
using (var visionClient = new ComputerVisionClient(credentials, handler))
{
try
{
imageStream.Position = 0;
visionClient.Endpoint = "https://eastus.api.cognitive.microsoft.com/";
var result = await visionClient.AnalyzeImageInStreamAsync(imageStream, features);
return result;
}
catch (Exception ex)
{
Debug.WriteLine($"{funcName}: {ex.GetBaseException().Message}");
return null;
}
}
}
}
}
And here’s how I referenced it from my content page:
pleaseWait.IsVisible = true;
pleaseWait.IsRunning = true;
var imageRecognizer = DependencyService.Get<IImageRecognitionService>();
imageRecognizer.ApiKey = AppSettings.ApiKey_Azure_ImageRecognitionService;
var details = await imageRecognizer.AnalyzeImage(new MemoryStream(ReptilePhotoBytes));
pleaseWait.IsRunning = false;
pleaseWait.IsVisible = false;
var tagsReturned = details?.Tags != null
&& details?.Description?.Captions != null
&& details.Tags.Any()
&& details.Description.Captions.Any();
lblTags.IsVisible = true;
lblDescription.IsVisible = true;
// Determine if reptiles were found.
var reptilesToDetect = AppResources.DetectionTags.Split(',');
var reptilesFound = details.Tags.Any(t => reptilesToDetect.Contains(t.Name.ToLower()));
// Show animations and graphics to make things look cool, even though we already have plenty of info.
await RotateImageAndShowSuccess(reptilesFound, "lizard", details, imgLizard);
await RotateImageAndShowSuccess(reptilesFound, "turtle", details, imgTurtle);
await RotateImageAndShowSuccess(reptilesFound, "snake", details, imgSnake);
await RotateImageAndShowSuccess(reptilesFound, "question", details, imgQuestion);
That worked like a champ, with a few gotchas:
I would receive a 400 Bad Request if I sent an image that was too large. 1024 x 768 worked, but 2000 x 2000 didn’t. The documentation says the image must be less than 4MB, and at least 50×50.
That API endpoint must be initialized. Examples don’t always make this clear. There’s no constructor that takes an endpoint address, so it’s easy to miss.
It can take a moment for recognition to occur. Make sure you’re using async/await so you don’t block the UI Thread!
Prettying It Up
Before I get into the results, I wanted to point out I spent significant time prettying things up. I added animations, different font sizes, better icons from The Noun Project, and more. While the image recognizer only took about an hour, the UX took a lot more. Funny how that works.
Mixed Results
So I was getting results. I added a few labels to my view to see what was coming back. Some of them were funny, others were accurate. The tags were expected, but the captions were fascinating. The captions describe the scene as the Computer Vision API sees it. I spent most of the day taking photos and seeing what was returned. Some examples:
My barista, Matt, was “a smiling woman working in a store”
My mom was “a smiling man” – she was not amused
Most of the time, as long as the subjects were clear, the scene recognition was correct:
Or close to correct, in this shot with a turtle at Petsmart:
Sometimes, though, nothing useful would be returned:
I would have thought it would have found “White Castle”. I wonder if it won’t show brand names for some reason? They do have an OCR endpoint, so maybe that would be useful in another use case.
Sometimes, even though I thought an image would “obviously” be recognized, it wasn’t:
I’ll need to read more about how to improve accuracy, if and whether that’s even an option.
Good thing I implemented it with an interface! I could try Google’s computer vision services next.
Next Steps
We’re not done with the app yet – this week, we will discuss how to handle the scoring. I’ll post updates as we work on it. Here’s a link to the iOS beta.
Some things I’d like to try:
Highlight the tags in the image, by drawing over the image. I’d make this a toggle.
Clean up the UI to toggle “developer details”. It’s cool to show those now, but it doesn’t necessarily help the target user. I’ll ask my mentee what he thinks.
Please let me know if you have any questions by leaving a comment!
Want to learn more about Xamarin? I suggest Microsoft’s totally awesome Xamarin University. All the classes you need to get started are free.
Update 2018-11-06:
The tags are in two different locations – Tags and Description.Tags. Two different sets of tags are in there, so I’m now combining those lists and getting better results.
I found I could get color details. I’ve updated the accent color surrounding the photo. Just a nice design touch.
I’m continuing my resolution to record as many of my programming and technical presentations as possible. I recently spoke at the inaugural Indy.Code() conference. It was excellent, with an incredible speaker line-up. I hope they, too, post some of their presentations online!
Should you write your app “native” or use a “cross-platform” solution like React Native, Xamarin, or NativeScript? The new wave of native-cross-compiling solutions provide significant cost savings, code reuse opportunities, and lower technical debt. Does wholly native, per platform development, still play a role in future mobile development? Let’s discuss together.
In this presentation, we’ll discuss:
The growth of native, hybrid, and cross-platform mobile development solutions
Cost analysis of multiple native and cross-platform apps
Considerations for each native and cross-platform solution
Many developers have had that “labor of love” project – the kind that keeps them up nights trying to get everything right, figuring out how to pass that one last hurdle. Woz was no different, and the recently open-sourced code – for non-commercial use, of course – brought back memories of the days he worked on it so long ago, finishing in Vegas no less.
Some of you know I used to work for Steve, so I reached out to him with a link to the his code…
Here’s his response:
On Nov 13, 2013, at 8:04 PM, ʞɐıuzoʍ ǝʌǝʇs wrote:
The MOST AMAZING code of my life…I could never do anything close to this much ‘out of any box’ stuff ever again…it was as amazing to come up with it as it seems to be reading my code. In some places I put numbers like (5) meaning that 5 cycles would be taken by that instruction – I had to count them all so the loops always sent a byte to the controller every 32 microseconds exactly. And there is no way to explain the 5-bit and 7-bit stuff but it extended the data from 13 sectors to 16 sectors. The 13-sector version was running in Las Vegas. The improvement to this 16-sector code is the part that I worked on every night for a month, nearly finishing each night around 2 AM (Denny’s milkshake) but repeating the whole process the next day because I had to keep getting the entire huge framework in my head each day. Finally I stayed one night until 6:30 AM and got it totally done. Jobs had been asking me every day when it would be done and that morning I told him that it was! This part of the low-level disk code was not Randy’s but I am so thankful for the parts he did so well too that made higher level sense out of this. I consider this code to be more like hardware than software.
Auri's Blog 