Raymarine MFD apps three ways: from drone control to sat comms to video streaming
MIBS 2018 saw the manufacturers putting the multi in MFD, with Garmin and Raymarine each introducing multiple new multifunction app and app-like display integrations interfacing to all sorts of third party boat systems, plus consumer electronics gadgets and online services — actually every single brand seen below, with lots more likely coming. The results are exciting, but what the heck are MFD apps? Fortunately, Raymarine is illustrating all three ways they can be created and why it’s useful to know how it’s done…
What’s an MFD app?
At the recent Miami International Boat Show there were three major categories of integrations on display. All three can end up looking pretty much the same when they’re displayed on your MFD but how they got there can be important. MFD manufacturers can use another company’s SDK (Software Development Kit) to control systems on the boat. Or they can publish an API (application programming interface) allowing third party developed integrations to be displayed on the MFD. Lastly, we also saw an MFD show off its ability to run off-the-shelf Android apps.
The first integration, an MFD reaching out to a third party via their SDK, takes the most work for the MFD developers but can result in the most customized and tightest integrations. An early example was MFD developers each coding to Fusion’s FusionLink audio control API; the interfaces are each different and often include difficult integrations like an onscreen sound bar. In Miami, Raymarine introduced the impressive SDK-based UAV (unmanned aerial vehicle, or drone) control detailed below.
The second type of integration, where others code to the MFD’s API, is actually a series of dynamic HTML5 screens being rendered on the MFD. The plotters able to display these integrations are running either Linux or Android, both of which have native HTML5 rendering (Android’s core is a modified version of Linux). These integrations require third party hardware on the boat that serves up the HTML5 via Ethernet or WiFi, like this Lumishore Lumi-Link Bridge.
HTML5 integrations — Garmin calls them OneHelm, while Raymarine groups them as part of their new Lighthouse Apps — allow the MFD manufacturers to easily offer integrations with willing third party marine companies without having to do the development themselves. So, wow, the Mazu all-in-one satellite communications system will suddenly be accessible on a bright Axiom screen — when Lighthouse 3.4 releases, hopefully this spring — and the Seakeeper monitoring app is coming to both Garmin and Raymarine. But there are limitations to HTML5 integration we’ll discuss below.
The last option is native Android apps and is currently unique to Raymarine (though Furuno TZT2 and Si-Tex NavStar MFDs also run Android). While these apps will be Raymarine approved and only available from their Lighthouse app store, a product manager told us that the Netflix and Spotify apps they demonstrated at MIBS were running without modification. That means nearly zero effort on Raymarine’s part while the Axiom user gets access to apps from big time services that would never build apps for special marine hardware. Again there are limitations — like the MFD must be online for music or video streaming — and there’s also the question “Is this taking marine navigation electronics too far?” But please consider the analysis below before making a judgement.
SDK “App”: deep drone control
First let’s deal with some possible confusion. When Lighthouse 3.4 arrives and you tap that new Apps button on the Home screen, you will not see Ray’s new drone interface on the new Installed Apps screen. These are the optionally installed “Lighthouse Apps” discussed below. But then again Raymarine has long said that the “LightHouse 3 home screen offers bold and simple app icons.”
In fact, Raymarine has used, perhaps overused, the “app” term for even completely internal MFD functions like chart plotting since LightHouse 2 days, though in some cases the system icon represents an SDK-based integration with, say, a pan and tilt FLIR thermal camera. Other SDK examples are the Theyr Weather app built into LH2 r17 — especially interesting as it will appear in LH3 as a straight-up Android app (seen below) — and the impressive new drone app.
The two of us got to see a beta version of an Axiom Pro controlling that tiny DJI Spark drone, and then posed on the bow as the drone streamed HD video to the Axiom over WiFi. That would be pretty cool in itself, but note how the chart window is showing the Spark’s location relative to the boat, as well as its “heading” (camera direction), and that’s just scratching the integration surface.
The context menu above suggests the many levels of drone control possible. Free Flight mode, for instance, overlays the streaming video screen with controls similar to the Spark’s optional joystick controller. But because DJI has developed an ActiveTrack mode able to follow a moving object, it’s also possible to have the drone auto fly in tandem with your boat capturing that underway footage you always wanted. In fact, the planned Fish On mode will purportedly combine several Spark features so you can keep fishing as the drone takes off and films the action while circling your tracked boat.
Combining a camera drone with an MFD is also useful for finding fish, as Raymarine was able to demonstrate in their Miami demo video, and future UAV app possibilities include flying to a charted route. Couldn’t that be useful before entering an unfamiliar gunkhole? And if you’ve ever tried to fly a drone using a small, dim display in bright daylight, you know how much easier it will be using a big bright Axiom MFD.
The bigger picture here is that Raymarine is working hard to integrate Spark (and even more feature rich Mavic) drone control to this level, but that’s what it takes sometimes to integrate with a complex product from an industry working at an entirely different scale. DJI alone has 2.7 billion dollars in annual sales — roughly double the entire global recreational marine electronics industry — but the Ray developers could go to the DJI developer site and get to work. Having Android in the Axiom helps, but this SDK-based app development is still in sharp contrast to the easy way HTML5 browser apps can add system integration.
HTML5 apps: Mazu sat comms
After the 2017 Miami show Panbo highlighted the SkyMate Mazu/mSeries all-in-one Iridium sat comms system, and also discussed an earlier version of SkyMate hardware that interfaced with the Android browser built into the Si-Tex NavStar MFD (good demo video here). The Mazu/mSeries has evolved a lot since, including a browser interface in addition to the dedicated iPad Mazu app, and that’s what you’re seeing on the Axiom LightHouse App screens above and below.
So suddenly an mSeries hardware user will have big, bright, waterproof screen access to satellite email and much more at the helm of a bluewater sailboat or up on the flybridge of a hard charging fishing machine. And it will be fairly easy for the companies to pull off because the mSeries can already serve HTML5 over WiFi while the Axiom MFDs can be WiFi clients (as well as WiFi access points) and display HTML5. While it will take a visit to Raymarine’s yet unseen app store, that also means that Mazu support will not burden the Axioms of boaters who don’t use the hardware and service.
But without a lot of extra development work, HTML5 apps like this will not integrate deeply with an MFD. You won’t, for instance, get a pop-up on your Axiom when a new message arrives (though the mSeries includes neat LED notification).
Another example is Mazu/mSeries weather. While the system can provide a surprising wealth of data given its limited Iridium connection, the NEXRAD radar images and/or GRIB wind/wave predictions will probably not overlay on the Axiom’s chart windows like they can in many MFD implementations of SiriusXM Marine satellite weather. The latter are SDK-based MFD functions while browser apps are much more siloed.
When we take a close look at Garmin’s several OneHelm partnerships we’ll be on the lookout for more HTML5 browser app limitations, but you can see already that the ease of implementation means that Garmin is suddenly supporting four different digital control and monitoring systems (Maretron included though not yet on that web page).
Android apps: Netflix & Theyr
Here again is the beta version of the LightHouse 3.4 Apps screen we saw in Miami. What you see are HTML5 delivered apps like Mazu — along with the SeaKeeper and Lumishore apps we’ll detail in the OneHelm entry — and several Android native apps packaged up for the Lighthouse store and ready to run on a plotter.
If you think about browser apps you may already use on your boat — like this old WaveWiFi screen — or cruise the Google Play Store for apps you’d like to have on your Axiom, it seems predictable that the LightHouse app store will eventually offer many choices and some users may fill that screen with installs.
The streaming entertainment apps like Netflix and Spotify will probably cause the most harrumphing from those who feel that navigation electronics should only be about navigation. But again, no Axiom owner has to get near these apps if they don’t want to, and they may change their minds when they realize that it’s relatively easy to get an Axiom online through your phone and watch “the big game” or whatever on a much better screen. We saw it working in bright Florida sunshine, and the Axiom could even drive Bluetooth speakers.
While the entertainment apps deliver a nice optional benefit, the real magic might be the ability to run a myriad of boating apps. Note, for instance, how the Theyr Weather screens on the Axiom above are the same as the Theyr GRIBview Android app. It’s true that this weather service, like Mazu, won’t integrate deeply with the MFD, but it’s also true that the LightHouse App store may eventually offer a dozen weather apps while the Ray developers are free to work on core projects plus goodies like drone control.
Overall Raymarine’s new ability to deliver a broad catalog of apps without having to develop anything themselves puts the company in an enviable position. But the step change to Android-based Axiom MFDs caused some pain for owners of previous Ray MFD models, and the LightHouse Apps feature adds a little more. In short, they are not going to work even on older MFDs like the eS and gS series that can run Axiom-like LightHouse 3.3.
It turns out that while the Ray developers built an entirely new MFD operating system on top of Android for the new Axiom machines, LH3 for the older eS and gS series is a different architecture (aside from running on less powerful processing hardware). So Axiom MFDs will continue to get new features that may not be backward compatible even to eS/gS, and LH2 will no longer be developed.
If you own a Raymarine a-, c-, e-Series or older, it’s fair to note that Ray added a lot of features to LH2 over the years, and they are still struggling to put some of those features into LH3 (check the FAQs). Plus, there’s an argument that SDK-based features like LH2 Theyr weather are better than the Android app version. But let’s note that the pace of change in electronics makes for a real challenge in balancing new hardware and software with supporting the legacy equipment. Don’t believe us? Just look at the comments section of the Raymarine Axiom Announcement from last year’s show.
But after massive amounts of Axiom Lighthouse 3 development while enduring some flack from LH2 users, Ray is coming out of the step-change tunnel. You could even say that the Raymarine product folks at MIBS 2018 had noticeable pep in their step; they had some exciting stuff to show off and they knew it. Between the apps, the drone control, and the Quantum 2 doppler radar (we will cover), there’s no doubt about Raymarine’s ability to move forward.
Sorry to add to the “MFD App” confusion, but I’m not the only one who has also used that term for a phone or tablet app that connects to an MFD… and Raymarine has new one of those coming too!
We didn’t see it in Miami but…”The new Raymarine LINK mobile platform for iOS and Android allows Raymarine users to plan, sync, and control their Axiom navigation display from their mobile device. For example, Raymarine LINK allows users to plan waypoints and routes while away from the boat. Once onboard, LINK synchronizes waypoint and route plans automatically. Raymarine users can also access trip logs, screen shots, and video recordings from the Raymarine LINK mobile app. Review trip logs at home and share trips, screenshots and video captures with friends. Raymarine LINK also provides backup for all MFD settings and keeps your Raymarine gear up-to-date with the latest feature and software updates.”
Sounds a lot like the Garmin ActiveCaptain app?
Hi Ben, besides the usual declarations, is there any evidence of decent online security involved? To say I’m uninterested in having some pimple-faced punk in Riga rummaging around in my primary navigation tool is a bit of an understatement!
Just need a smartphone with 800-1000 nits to use in direct sunlight. Looks like Galaxy S9 is close (700 nits)
Hartley,
Raymarine didn’t have anything to say one way or another about security (and I should admit we didn’t ask). The underlying Android OS’ security is likely to be the primary information security involved. Your opinion of that security may vary but I do think it’s likely there’s a good deal more sensitive and interesting information to a hacker on your smartphone than on your MFD.
Ben S.
Hi Ben S! Well, yes and no – while my phone’s contents might be of more interest to the thief, any sort of computing device is of interest to someone building a bot net — which is why the Internet of Things (IoT) is becoming such a mixed blessing.
I doubt any hacker is interested in my routes and tracks (though they might be if I was someone important), but they might find it amusing to mislead my autopilot or alter my variation setting. Worse, it might just slow my MFD down so much I can’t tell where I am some foggy night in Maine..
Here’s an article that might provide some food for thought:
https://www.wired.com/2015/12/2015-the-year-the-internet-of-things-got-hacked/
73 DE Hartley
Hartley,
I stand by the notion that I have more critical data on my phone than my MFD. I don’t dispute the notion there’s nuisance / amusement value to a hacker being able to remotely control navigational electronics.
As far as a bot-net, in most vessels I believe the MFDs will be sporadically connected in a manner that doesn’t make them overly useful in this capacity. My boat is in fact a little different in this respect in that it is connected nearly always. As a result, I employ an enterprise-grade router giving me diagnostic data on the end points connected, their remote connections and data transfered. Network level AV and anti-malware is an option as well. Like any other network security is obtained through defense-in-depth. Attempting to secure the devices alone isn’t enough.
Your questions cause me to circle back to one of the questions I’ve asked of Raymarine but not yet gotten an answer on. How are apps added to the list of approved apps for the MFDs? If Raymarine is curating the list of apps that can be installed is part of this curation based on security reviews?
Security is a highly valid concern. You raise some great points and it’s something we will need to ask the MFD manufacturers about as they increase the connectivity of their devices. For me personally, security of these devices hasn’t yet risen to a point that it would slow my adoption. Given the way these devices have been implemented and my use of them I’m comfortable with the risk. YMMV.
Cheers,
Ben S.
Great to see marine manufacturers leveraging mobile technology to deliver some exciting new capabilities, but where is Furuno? I invested in the newish (and Android-based) TZtouch2 MFD…should I remain hopeful that they, too, will join the app parade? They seem to be a half-beat behind in delivering these kinds of innovations…
Hi Again, Ben S! You are hopefully correct about the limited nature of the Internet connection – but it seems to me that most of the “new capabilities” being touted here will require a high duty-cycle connection to the Internet – certainly the AIS, weather and NEXRAD aps will require a connection to be useful, not to mention the “entertainment” aps – including “watching the big game” 🙂
It is this trend of invoking more and more connected features that concerns me – UNLESS the security concerns are addressed along with them – hence my concern.
With my LH2 stuffe, I’m pretty unlikely to be connecting my MFD to the Internet – but when the time comes to move forward, this WILL be a concern (especially if updates and refreshes will require an Internet connection vs today’s micro-SD card “sneakernet” approach).
73 DE Hartley
Another informative article. While I’m all for well thought out APIs and good SDKs I have the following concerns:
1) APIs need to be right from the beginning. Reworking them later breaks backward compatibility and leads to abandonment by third parties
2) App stores are filled with crapware. This includes everyone’s app stores and not just Android. For a mission and safety critical system this would imply an open source mandate, an independent validation of ‘allowed’ apps and some strategic guidance by governments looking to the longer term.
3) I can see the marine hardware vendors business models needing to mandate monopolies and frequent obsolescence. All factors we don’t want.
4) Open source software, OS’s and firmware is the only trusted approach to software defined things. (proprietary code has no place anywhere IMHO) It allows third party and independent validation and maintenance when vendors move onto the next shiny thing.
5) No mention of backward compatibility to current hardware. Where does B&G approach fit into this. The GoFree app store is a sad, sad thing. Even their ssl certificate is out of date. This doesn’t make me want to trust these folks.
6) Much of the crapware in the app stores relies on good, fast, always connected internet connections so the app can be overly chatty to the mothership. This behaviour just consumes valuable onboard resources for no benefit to the operator. What standards are proposed to ensure security, performance, robustness and low enery consumption are optimized? My first reaction after reading your great article is no way do I want any third party crapware on my boat.
I look forward to following your and your contributors comments.
Thanks, Ian. I don’t know enough to comment on most of your points, but I am dubious about “marine hardware vendors business models needing to mandate monopolies and frequent obsolescence.”
Obsolescence certainly happens in marine electronics but the driving forces I’ve observed are the attractions of better suited operating systems — like Ray’s painful switch to Android — and improved networking protocols that aren’t necessarily backward compatible with existing sensors etc.
I’ll add that while my new colleague Ben Stein is much better suited to discussing your concerns, he’s currently deep into rebuilding the Panbo website. But you can help him get free from that project by giving us some Panbo reader feedback 😉
https://panbo.com/forum/2018/03/new-panbo-website-whats-good-whats-bad.html
Raymarine LightHouse 3.4 is out:
http://www.raymarine.com/multifunction-displays/lighthouse3/v3-4/
Sorry to add that a couple of the apps discussed in this entry, like Seakeeper and Mazu, won’t be supported until a “future software updates” but then again 3.4 includes enhancements that weren’t demonstrated in Miami.
(And owners of Raymarine MFDs that will not get LH3 may appreciate the fact that some of 3.4 enhancements are already in LH2.)
Just finished writing up my notes on v3.4 and the various features within at https://sailbits.com/raymarine-lighthouse-v3-4-apps-bluetooth-find-nearest-and-more/
I think apps are a great new feature that could drive some interesting integrations. The rest of the features are great too, including finally being able to control brightness from one place. Now if they would only add a “night mode”….
Wondering what happened to the Axiom drone integration feature? Me too. I don’t think it’s been included in an update yet, but Raymarine is demonstrating it again at iCast so I guess it’s still on its way:
http://www.icontact-archive.com/jjiXoF3Cf8NfV-NNtAkNpeh3OOjJaIsb?w=4
Lest our readers get too excited about the prospect of running ‘off the shelf’ Google Store apps on the Raymarine Axiom series MFDs, you can’t.
As of v3.11.42, if the app isn’t ‘signed’ by Raymarine, Lighthouse 3/Android will not install it
My Axiom is used for situational awareness (and updating the autopilot components).
Watching Netflix on an 800×480 screen was never one of its missions.