Panbo

AC34 LiveLine, harder than football & open sourced

... written for Panbo by Ben Ellison and posted on Feb 6, 2012
AC34_LiveLine_in_Plymouth_UK.jpg

Never mind the final Super Bowl score (and the unfortunate headlines), did you appreciate those slick digital field overlays like the yellow 1st & ten line? Frankly I'm still impressed with this technology even though it's not magic. HowStuffWorks explains pretty well the sensors, geometry, and video processing required to make that virtual line look like it's painted on the field. All it takes is precise knowledge of where the camera and line markers are in a 3D model of the stadium, plus the use of the turf color as a reverse mask so that the players -- who must never wear the same shade of green -- don't get overlaid. But once you understand that, the new LiveLine info inserted into America's Cup video seems practically impossible...

Consider the leap in sensor difficulty between tracking the location, pan, and tilt of a camera hung from a helicopter as opposed to those set on tripods around a stadium. Consider too how much larger the AC34 playing field is, which means that even small errors in camera or race/mark boat tracking can result in wacky overlays. And finally consider the communications and processing horsepower needed to make LiveLine live. No wonder than the seemingly endless gear racks and networking ganglia I spotted in San Diego around the complex of containers at the heart of what's known as ACTV...

AC34_ACTV_tech_cPanbo.jpg

I've been intrigued with LiveLine since I first heard it about last June, and learned more about the underlying high-precision telemetry when I got to San Diego. So I especially enjoyed lurking around the LiveLine booth (container) during an afternoon of the racing.
   Riding herd over all the technology are some talented humans, and the photo below shows the LiveLine chief operator at work. The gray switch box near his left hand lets him manage audio coms with the helicopter pilot and other nodes in the system, including the adjacent broadcast booth where LiveLine is mixed with the many other video feeds. Meanwhile the nearest PC screen shows the software he uses to select specific LiveLine overlays while the monitor above it shows the results and the one top left shows the mix going out on live ACTV. There was only minor grumbling about how the director should use more LiveLine!...

AC34_LiveLine_chief_operator_cPanbo.jpg

Then there's the mask man. The color of harbor water is not nearly as consistent as, say, AstroTurf, especially given the changing light angles from a helicopter, so his job is to constantly monitor the shades used to create the overlay mask. If you click the photo below larger you'll not only see how perfectly all the boats and land features are being excluded from the overlays, but also a window of "LiveLine Chatter Feed".
  AC34 seems to have numerous communications levels, including several text streams, both to accommodate those with limited screens -- like the racers, and the umpires on Jet skis -- and to serve as backup if the big pipes fail. In fact, the reason Stan Honey's laptop below was running AirView Spectrum Analyzer was because on this particular afternoon some unknown RF source was intermittently interfering with the wireless video feed from the LiveLine helicopter. Note that some of the same experts who live simulated previous Cup races to make it easier to understand on TV are working at the back of this same container, so LiveLine is even backed up graphically...

AC34_LiveLine_mask_operator_cPanbo.jpg

ACRM (remember how the umpiring and course management use the same underlying telemetry) and ACTV test all systems every race day morning, hoping to avoid such surprises. And the LiveLine team wasn't happy about the distorted copter feed, so Honey did a lot of troubleshooting, including calls to the big Navy base across the harbor and to a cruise ship that had just parked in a nearby berth. I felt weirdly lucky to witness how well he and the team worked the problem, but in fact it went away by itself. I wouldn't be surprised, though, if a pro-level directional antenna and spectrum analyzer gets added to the gear kit...

AC34_LiveLine_container_Honey_spot_cPanbo.jpg

I was also pleased to meet Ken Milnes, the gray haired gent sitting with the LiveLine crew. He and Honey founded Sportvision and developed that yellow 1st & Ten Down Line together (and many other sports broadcast enhancements),  and he now serves AC34 as Live Graphics Project Manager. It's Milnes who developed the Streaming Data Interface Specification that's intended to let developers access AC34 telemetry, chat streams and more in real time.
   Yes, you read that right. Developers are free to write to write apps that might, say, put LiveLine style graphics onto the smart phones of AC spectators lining Newport or San Francisco harbor, or even overlay it on the phone's geo-located camera display. Logged data from San Diego and prior races is already available, and it's interesting to note that there's also a Specification for streaming the data to the race boats, presumably the 72-foot version. There's even a sample ACViewer app which can interface with a AC-served sample San Diego data that works right now (I tried it). Who's going to cook up some AC34 apps and what might they be besides enhanced viewing apps?
   The more you look into what AC34 is doing, the harder it is to deny the claim that sailboat racing is being reinvented before our eyes, with the AC World Series in particular perhaps becoming an enduring professional circuit. I'll close with a peek into the main ACTV broadcast booth (container) while Oracle Racing CEO Russell Coutts makes the case to some San Diego visitors.

AC34_ACTV_container_w_Coutts_n_VIPS_cPanbo.jpg

Comments

What is really cool about the lines and logos laid on the TV is that--as I understand it--they are the result of the innovation of Stan Honey, the same guy that in partnership with Jim Corenman established Sailmail and contributed to Winlink. Those services and the Airmail product that connects to them provide e-mail over HF/SSB radio to thousands of cruising sailors and boaters. Go Stan!

Posted by: Auspicious at February 8, 2012 11:24 AM | Reply

Thanks for mentioning that, Auspicious. Honey is even credited as "organizer" on the SailMail site:

http://myboatsgear.com/mbg/index.asp

And the guy was just awarded Rolex Yachtsman of the Year for his role as navigator aboard an otherwise all French trimaran that set the non-stop circumnavigation record (in 48 days!): http://goo.gl/sa5mx

I would have made more of Stan in this entry except that I already got pretty carried away last June (and I don't want to make such nice man nervous ;-): http://goo.gl/soeRp

Incidentally, the guy in charge of AC34 fleet electronics is Eric Steinberg, who also runs Farallon Electronics, which may be the biggest distributor of Pactor modems in the U.S. He also developed the iStar GPS (with help from Stan) that I wrote about here: http://goo.gl/SkdOl

Eric was a huge help to me in San Diego and he will figure in a future entry about the tricked-out Jetskis the umpires use. The depth and breadth of talent behind AC34 is amazing, and I'm afraid I've become a total fan boy.

Posted by: Ben at February 8, 2012 1:18 PM | Reply

Ben, you are one of a legion of fanboys. AC has entered the 21st century with technology and opportunities that could not have been imagined a very short time ago. It's a godsend to the sport that Silicon Valley won. NASCAR looks tame. It will never impact my personal enjoyment of participation in the sport in my 30+ year old boat, but the nonstop Internet coverage has brought a real lift to my winter.

Posted by: Steve Hayes at February 12, 2012 6:34 PM | Reply

AC34 just put up some good videos aboat how LiveLine is created:

http://www.youtube.com/watch?v=r0LH5cCuc_4

...and what it looks like:

http://www.youtube.com/watch?v=Bf43WZtdyEY

Oddly, though, they're both based on video footage taken before the San Diego event. Much has improved. I've even heard that KVH has supplied an specially adapted tracking antenna to beam in on the signal from the LiveLine camera helicopter.

AC34 has also cut a deal with NBC to broadcast the racing. It's an unusual deal, though:

http://alamedawaterfront.com/stories/how-to-watch

Posted by: Ben at March 17, 2012 10:51 AM | Reply

Pleased to report that Yachting has already posted most of my April issue feature about AC34:

http://www.yachtingmagazine.com/electronics/ac34-demands-your-attention

Missing are the sidebars about Stan Honey and the new Mark/VIP boats, and -- don't know why -- my own photos.

Posted by: Ben at March 17, 2012 11:13 AM | Reply

LiveLine got an Emmy nomination! http://goo.gl/ZPa0h

In other news, racing starts in Naples on April 11 and I'm about to join a conference call with Team Oracle

Posted by: Ben at March 30, 2012 11:55 AM | Reply

Stan Honey spoke at MIT Sloan Sports Analytics Conference, and though the boobs who videoed it left out his presentation, this is a terrific explanation of LiveLine and where it's headed. Stan says that they may even overlay graphics that show the bad air created by a racer's sails. He also confirms AC34's intent to make all the data available in real time to all developers.

Posted by: Ben at March 30, 2012 4:43 PM | Reply

Hard to work this morning with live AC45 racing from Naples going on YouTube. Gary Jobson is narrating, LiveLine and onboard cams/mics are working great, and there's LOTS of wind and sea:

http://www.youtube.com/americascup

Posted by: Ben at April 11, 2012 10:42 AM | Reply

Leave a comment