Location Manager for MSFS – The Challenge of Enhancing the UI/UX
Following on from my previous post about Aircraft Manager for MSFS – Development Overview , here is a short overview of how the development of Location Manager panned out.
To recap what the problem is: MSFS is like an interactive earth simulator, enabled by flying around complex mini-simulations in the form of aircraft across the whole history of flight. The two methods for currently managing locations in MSFS are show below.
Inbuilt MSFS method 1 for location management
Inbuilt MSFS method 2 for location management
But there’s no way to save and/or favorite locations in the sim itself. While you can save locations outside MSFS in all sorts of ways (OneNote, LittleNavMap, Excel, etc) it’s not the most convenient of methods if you want to build up a log of world exploration discoveries for easy access within the simulator. This isn’t the same as loading a saved flight plan either. There are many use cases where you don’t actually want to use a flight plan, but just go to a location and start flying from there. So the intent here was to derive a solution to solve this problem.
Step 1 – Validate the Technical Approach
First off, just as in the previous post, I needed to validate some technical issues before diving into the prototyping stage. This was a very different project from Aircraft Manager, as that relied mainly on the inbuilt aircraft database supplied by the sim. In the case of Location Manager for MSFS, one of the big issues here was we needed to develop a robust way to save locations with associated user data via the inbuilt MSFS UI storage system. There were also unanswered questions as to how robust this storage system is in terms of how much data can be safely stored, on both PC and XBox. I have received some advice from one of the Asobo devs on this (thanks!) but still need a little more clarity, so need to keep an eye on this.
In summary, some of the biggest technical issues to be validated were:
- How to capture and save the location information from the MSFS UI
- Where would be the best place to insert the UI to manage saved locations
- How to replicate the map zoom functionality in MSFS for saved locations (this was tricky!)
- How to efficiently store locations for easy retrieval, display, updates, and deletion
So I got to work, solving each of these unknowns in turn, or at least getting me into the ballpark of “this looks doable in a performant manner so the projects not dead and I can keep going”. Like the previous project, there were a number of dead ends and frustration moments which necessitated some creative thinking in order to spark the idea for another way to solve the problem. Some of these solutions come from gaining a deeper understanding of the MSFS UI framework, some from creatively applying workarounds. A workaround is a little different from a hack btw. A hack is typically a short term fix, designed as a purely quick and dirty temporary measure while you search for a better more reliable long term solution to a problem. A workaround is a solution you have confidence in being robust enough for long term use, even though it’s not how you originally envisioned it working. Workarounds are always preferable to hacks, giving you better maintainability over the long term.
Step 2 – Validate the Prototype
After gaining sufficient confidence I had working solutions to the core technical issues, I could start constructing a full prototype incorporating these into a working demo. This lets you test & tweak the full workflow, and weed out other issues/bugs.
Early Prototype of the Map interface
Early Prototype of the location management interface
Location Manager for MSFS Free Demo
The next stage from this is to get a demo into peoples hands to get wider feedback, while continuing to tweak workflows and solve issues/bugs as they appear.
Similar to AM, I fired up a thread at the Flight Simulator forums to see if anyone is interested in trying it out. Have at it, all feedback is welcome: The Good, The Bad, and The Ugly.
You can download the free demo lite version from the Location Manager for MSFS project page. Like the AM demo, it’s fully functional for casual use cases, with no obligation to upgrade. Enjoy!
Location Manager for MSFS Pro Release
Version 1 of LM Pro
I’m currently in the final stages of release for the pro version of LocationManager for MSFS.
On the roadmap for future enhancements are:
- Import/export of location data (can also use the export for Aircraft Manager)
- Toolbar widget to list saved locations and teleport to them
- [insert here]
Not sure if these are possible at the moment, I need to do some technical validation as per Step 1 above. We’ll see!
If you got this far, thanks for reading. Stay tuned for more updates in the future.
Enhancing the MSFS UI & UX to solve the Aircraft Management Problem.
MSFS (Microsoft Flight simulator) has been a going concern for 40 years, with some ups and downs along the way, and the latest iteration launched in 2020 is revolutionizing the flight simulation space. It’s a complex simulation for sure, not without issues or growing pains, but it’s undeniable that when it works well it is an amazing experience in either flatscreen or VR. I’ve been using it extensively on and off since the launch, and while I like the UI there’s always been a few niggling issues with the main interface that I’ve been itching to scratch. One of the pain points is with the Aircraft Management aspect via the Aircraft Selection Screen, and Aircraft Manager for MSFS is my solution to that. This blog post is a short story of how that came about.
MSFS Aircraft Selection Screen
A couple of times in the past I’d taken a look through the User Interface code along with exploring the official SDK, but as there was no documentation on it I steered clear of doing anything with it because it looked too complex. Diving into a complex codebase with no documentatioj to guide you can be a bit of a scary prospect, and you need to be prepared to persist with it in order to get that series of lightbulb moments where you being to understand it.
Anyway, towards the end of 2022 I was casting around for new side project ideas that could solve a real problem and make an impact. Two areas in MSFS stood out for me as candidates to develop something for:
- Aircraft Selection and management, made more difficult with the growing number of aircraft now available both paid and free.
- Managing locations inside the sim. MSFS is like an interactive earth simulator, enabled by flying around complex mini-simulations in the form of aircraft across the whole history of flight. But there’s no way to save and/or favorite locations in the sim itself.
So I decided to look at enhancing the Aircraft Management problem first. More about Location Management in a follow-up post.
The Problem To Solve
Some of the issues with it are:
- A growing aircraft collection means too many planes, and it’s easy to get overwhelmed deciding what to fly using the current UI
- Frustration you can’t tag your aircraft the way you like
- Can’t sort or filter them in meaningful ways to use them in various contexts eg: racing, stol, vfr etc
- Can’t take easily accessible short notes ingame as reminders before flying
- [insert your itch here]
First Steps to Developing a Solution to Aircraft Management for MSFS
Getting to grips with the MSFS UI (User Interface) was the first big rock to move, as we say in project management. You can’t cut into a codebase unless you know what you’re doing.
Unfortunately, there is no documenation on the UI at all (some is finally coming in SU12 release, though) so it meant doing it the hard core way: reading and debugging, piecing it together line by line.
Fortunately, this is one of my core skills as a developer. I’m a regular bug magnet, so my debugging and analysis skills are fairly well developed. That said, I knew it wouldn’t be easy, which is why I’d avoided it so far!
So I blocked some time out and I went for that long overdue deep dive into the MSFS UI codebase. Three days later, after some hair pulling, screaming, wtf’s, and aha’s, I surfaced with *some* idea of how it was put together. Shoutout to both Microsoft and Asobo for providing the source code in the first place, and also to the dev(s) that architected and implemented it. Impressive framework effort, and not an easy project to develop even leveraging a UI library.
My pathway to developing this was through iterating rapid prototypes to learn more about how the MSFS UI framework worked, while testing out ideas for how I might enhance it, before launching into the actual development solution. This next phase was even more educational, as I ran across the “gotcha’s” inherent in an undocumented system built on top of Game UI library Coherent GT.
Not having the ability to implement anything at the Coherent GT level meant understanding fully what might be possible to do, or not, and finding the best solution. I had a short list of technology solutions I wanted to use, based on my previous experience, and prioritised them into a risk reduction approach to prove various technical issues could be solved in order to develop a working enhancment solution.
I worked my way through various ideas I had, and was getting a little dejected when they all ended up on the cutting room floor. This project involved lots of creative pauses, while I had to go for a walk and let it stew for a while in order to spark an idea to find another solution to try. Eventually, I hit paydirt with a robust solution I could build upon that seemed reliable enough to surgically insert into the MSFS UI system and enhance it without disrupting regular use, a pretty critical requirement!
Developing the Prototype
Prototyping a more complete solution was the next stage, where you take the bits you now know work reliably and start to craft a proof of concept that covers the core use cases. The goal of the product development wasn’t to develop an overly complex solution, but a lean mean aircraft management machine, that solves a couple of core use cases very well, as opposed to being a swiss army knife solution.
Once I had a working prototype that was functional enough to prove the concept I reached out to other MSFS users at the MSFS forums to see if anyone was interested to test it and give me some feedback.
Early Prototype of Aircraft Manager for MSFS
Shoutout to the following users for taking a chance on testing and providing feedback for Aircraft Manager for MSFS:
It’s always vital to touch base with other users, as even though I’m scratching my own real itch with the UI and solution other users have different itches, and sometimes they overlap in interesting ways. Stress testing for performance is also critical, as is squashing bugs which are easy to miss even with a QA checklist.
The end result (but not quite the final one, it’s changed already!) you can see in the video walkthrough below:
AM Free Demo
You can download the free demo lite version from the Aircraft Manager for MSFS project page. It’s fully functional for casual use cases, with no obligation to upgrade. Enjoy!
AM Pro Release
Version 1 of Aircraft Manager Pro for MSFS
I’m currently in the final stages of release for the pro version of Aircraft Manager for MSFS. Making a product is only part of the challenge, there’s a lot of other elements that go into a successful rollout, open source, free, or paid.
This stage always reminds me of the 90/90 rule:
The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.
—Tom Cargill, Bell Labs
It’s not just code. Website, payments, fulfillment, support, maintenance and the rest all need to be done. That’s a lot, and I found I had to refresh a few of these areas as well, which hung up the deployment of the pro version.
The Story Continues: Location Manager for MSFS
As I mentioned at the start, the other problem I came up with to solve was:
- Managing locations inside the sim. MSFS is like an interactive earth simulator, enabled by flying around complex mini-simulations in the form of aircraft across the whole history of flight.
After getting hung up on some of the product deployment issues, I decided to take a look at this problem while waiting on people to get back to me.
Location Manager for MSFS free version is now available from the project page,
I’ll be writing more about this shortly in a follow up post. Stay tuned, and thanks for reading if you manged to stick it out this far!
Update: Part 2 is now available at Location Manager for MSFS – Development Overview.
The sound and music are 50% of the entertainment in a movie. George Lucas
As it is in games, I would also say the same for emerging mediums such as VR, AR, and MX.
I’ve been working recently on a couple of projects related to a few different areas, such as:
As part of this, last week I presented an introductory tutorial at the VRHubTokyo Meetup on Spatial Audio in Unity3D.
This is a really interesting area, with a lot of challenges still to be overcome.
This panel video of immersive audio experts is an excellent introduction to a lot of the issues:
We also had another Canvas Tokyo Musicians Meetup where we discussed a few of these topics, among others.
You can read the event report here: Musicians Meetup #2 Event Report – Mon 11th Sept @ Tacchi Studios
Check out the Spatial Music Visualizer which I also used to discuss a few of the issues around spatial music.
Stay tuned for more news soon!
Chatbot development and deployment
There’s a lot of hype at the moment about AI in general, and chatbot’s being the next big “app market”. Reality is the tech is still very immature and there are not a lot of use cases where the chatbot workflow is a better UX than simple point and click with a well defined menu and information architecture.
FAQ’s are an important corner of the customer support sector and is one area where they may have use so I decided to test out Microsoft’s bot service, which is still in preview mode: https://azure.microsoft.com/en-us/services/bot-service/
I created an FAQ chatbot for one of my own products as a practical test, which you can see running live here: http://www.harpninja.com/wp/faqhelp/
As you can see from the image there’s quite a bit to it, architecturally and technology wise, with a number of different services from NLU (Natural Language Understanding) to the bot development to connecting various user touchpoint services. There are a number of different types of bots you can implement and an FAQ bot is best implemented using another service they have called https://qnamaker.ai
Overall it’s a pretty nice developer workflow considering the different services they have hooked up to make the development and deployment a relatively painless experience. This FAQ chatbot is using their inbuilt web chat preview which made it a snap to integrate into WordPress without having to code up a plugin. The bot has also been deployed to Skype, Slack, and Facebook Messenger.
There’s a lot of different options for developing chatbots, which this article gives a great overview of: https://www.oreilly.com/ideas/bots-what-you-need-to-know , but I would definitely recommend Microsoft (the new Apple!).
Interactive Data Visualization & Exploration
I recently attended the Rakuten Technology Conference 2016 and saw a very interesting presentation from Erik Berlow titled “Next-Generation Interfaces for Thinking with Data“, which was about data scope interfaces.
I’m familiar with Data Science, data mining, and data visualization and have been working with a few tools, from RapidMiner to cloud based ML platforms such as IBM Watson, Microsoft, Amazon, & Google.
Data Scope Interfaces
Erik opened my eyes to some new ways of looking at the problem of constructing visualization and analysis tools in an increasingly data saturated world. He introduced the concept of Data Scopes, interactive tools for exploring multidimensional data and inter-relationships using the human visual system as the pattern recognition engine. The data can be pre-processed with Machine Learning to slice, dice, connect and categorize to provide a rich content source for the visual interfaces to explore.
Living in quake city (Tokyo) I decided to start with the NOAA Global Significant Earthquake Database, 2150 BC to present for this exercise.
As anyone who has worked with data science will tell you, massaging the data into shape is usually the most time consuming and painful part of the process. Garbage in, Garbage out as the old computer saying goes! I actually asked Erik in the Q&A what his processes were to clean the data, and he made the interesting comment that Data Scopes are also useful for rapidly detecting anomalous data. That’s why this earthquake data set is a great example because it has lots of missing data, in addition to lots of useful data.
Have a play with the Data Explorer below and see what patterns and relationships you can detect. See if you can spot where the data is missing, and see if you can spot where data looks a little strange (usually due to inaccuracy of records going back in time I suspect). It was a fun project and I learned a lot which I hope to apply moving forward to some more explorations.
Disclaimer: I did a small amount data manipulation for the purposes of this example . Many of the largest & oldest quake events are missing data in key categories, so I added a baseline 10000 to total injured in order to get some to display in the bubble chart.
Open the Quakescop in a new tab: https://sonicviz.github.io/quakescope/
Source code available on GitHub: https://github.com/sonicviz/quakescope