Panbo

ClearPoint weather, anybody else tried it?

... written for Panbo by Ben Ellison and posted on Nov 7, 2007

Clearpoint_screen_cPanbo

I’ve been trying ClearPoint High Definition Weather for a couple of weeks now, plus I talked to some of the team at FLIBS. This is a very ambitious operation. The ClearPoint goal is to provide the richness and simplicity of dedicated XM and Sirius satellite weather anywhere on the globe, delivered by Internet. Plus they’re aiming to provide more exotic data—like 1 kilometer wind forecasting of certain sail racing hot spots and game fish revealing altimetry/chlorophyll imagery—for certain niche marine markets. And they seem to be working with numerous nav software/hardware developers to provide integrated packages, even automated weather routing.
   At any rate, check out the full size screen shot of ClearPoint’s PC client software above. The icons at the top—wind, fronts, lightning, hurricanes, and text forecasts—show what I’ve chosen to display on screen. With the window lower right I can page through those various elements, clicking for specific places on screen. And the icons below show other data types I could add to my mix. I like the interface quite a lot and some of the data, like far offshore lightning, I’ve never seen in any weather product before. But buying the ClearPoint service is not trivial, and that does not include the cost of offshore downloads, which could be in the .5 meg per hour range (there is a low bandwidth mode, and ClearPoint is working on total download size controls). Interested? There is a free trial; please report back!

Comments

On the surface this seems like a very comprehensive tool. The user interface is very intuitive and the data well presented. It reminds me a great deal of the UGRIB interface from www.grib.us in the way it presents data - I guess it is because they both use a lot of .Net type controls. I have a couple of observations.

1. I did not seem to be able to control the amount of data I was downloading in terms of area, resolution or type of data. This is fine over my home broadband but less useful at sea. If I am in the middle of a transatlantic crossing then high-res wind is not so necessary and I might choose not to download satellite imagery everytime.

2. The second question is regarding the source for the high-res and very high-res data. Speaking to some experts in this field it is relatively easy to run higher resolution models within the standard NOAA (GFS) data but unless the inputs (boundary conditions) are augmented with additional local data (e.g. radar, soil readings, accurate topography, etc) then the results of the the high res model are often not much more accurate than the original GFS data.

Hope this is useful food for thought!

Posted by: Nat Ives at November 8, 2007 7:48 AM | Reply

Hi,

Agree with Nat. Control of content and bandwith of downloads is necessary. For racing, a .5 meg would mean U$ 17 using a Fleet77 service, wich is definitley not much for a racing team in a trasatlantic crossing, once or twice per day. For the global cruiser however, price becomes scary high.

Wonder how good is accuracy here in south american coast ... it is worth to try in the next races!

Posted by: Igor Stelli at November 8, 2007 8:20 PM | Reply

Useful food for sure, Nat!

1. There is a "low bandwidth" option in the setup menus, but ClearPoint has said that it intends to give users total control over what data is downloaded.

2. I definitely asked the question "Is the high res data simply interpolated?", and was told emphatically not. The local factor mentioned was topography, but there may be more.

Posted by: Ben at November 8, 2007 9:00 PM | Reply

Nat- some comments for your observations:

1. NAT- "I did not seem to be able to control the amount of data I was downloading in terms of area, resolution or type of data"... to answer this obersvation:

STUJ24: Current generation of the ClearPoint client is the "broadband" version, which you have seen. However, in the "Options" menu, you have the ability to select "Low Bandwidth", a 35 kb file with very usable data for offshore/ bandwidth restricted/ cost restricted requirements. NOTE- this "low bandwidth" was designed for use over Iridium phones data connection at 2.4 kbps (about 3 minutes air time). So, VERY cheap to get data updated every 6 hours.

2. NAT- "...the source for the high-res and very high-res data....relatively easy to run higher resolution models within the standard NOAA (GFS) data but unless the inputs (boundary conditions) are augmented with additional local data..."

STUJ24: Interesting observation. Ever wonder WHY the NOAA NWS and other commercial competitive providers of weather forecasting, particularly at local levels, are often inaccurate?? They ALL have a tremendous data management/ processing problem.

Here's why: (1) the NOAA GFS model is several generations "old" compared to the version ClearPoint is using; (2) the data outputted in GRIB 1.0/2.0 formats is "dirty"; (3) the process to aggregate weather data from multiple sources (e.g. "free/ public" goverment data, commercial/ private) and the "homogenization/ filtering" of data is NOT done by most weather providers known in the business; (4) the weather models themselves are classic "GiGo" (garbage in/ garbage out) and are highly susceptible to user/ error inputs and, therefore, output inaccurate forecasts.

"Free" data and "free" NOAA GFS models are just that, "free". You get what you pay for. Often innaccurate forecasts.

ClearPoint has benchmarked against many of the standard public/ private weather services (Weather.com, Intellicast, Weather/SailFlow, UK METS, France METEO, NOAA, etc) and often find that ClearPoint has a statistically significant advantage in accurate forecasting. However, I urge you to do the same comparison over a period of 30 days and let us know your conclusions, too.

NOTE- StuJ24 is ClearPoint's COO and has been an active racing sailor around the world at Regional/ National/ World-class levels.

Posted by: StuJ24 at November 8, 2007 10:59 PM | Reply

So Stuart, just to clarify. You run your own (later) version of the GFS model for the globe based on a higher level of data assimilation from various sources than NOAA and run it at a higher resolution for some chosen venues. I presume the very high-res data comes from an additional dedicated model with extra local inputs.

Nat

Posted by: Nat Ives at November 9, 2007 6:11 AM | Reply

Igor- some thoughts for you:

We recognize the need to "control" data downloads. See the comments earlier to Nat regards how this is being handled. We recognize the need to bandwidth "constrain" the pipe so as not to incur undue pricing/ cost issues for "world cruisers"- the price "inelastic crowd". As for "racers", generally price "elastic crowd", price is not a consideration, but "data" for decision weather intelligence is KEY!! Thoughts from you??

Posted by: StuJ24 at November 9, 2007 9:09 AM | Reply

Nat- no, we don't run a "later" version of NOAA GFS. For the purposes of clarification it's important to understand the following:

1. NOAA GFS is the basic data "aggregation" system for all weather worldwide. This data is used to initialize virtually all known numerical weather modeling worldwide. Why? Because US taxpayers are paying for the needs of our Department of Defense/ NASA/ NOAA and that means, quite simply, cooperating with both friend and "foe" to gather all known data worldwide in order to generate better weather forecasts. In addition, ClearPoint relies on very accurate commercial/ private sources to ensure ClearPoint ahs the most accurate weather data down to local/ regional sources (all proprietary).

HOW you use that data and WHICH data to use is CRITICAL to HOW the MODELS produce more, or less, accurate forecasts. Remember, GiGo...simple message??

2. NOAA GFS includes a baseline weather model that "anyone" can use "off-the-shelf". In other words, if you use the "GiGo" principle, the data you select is directly proportionate to the data you output using infinite "switches". Imagine an infinite input model, with infinite "switches" and with an infinite output model--- the ultimate physics/ mathematical "infinite model". That's the "state of the art" today.

3. There are more advanced weather models than NOAAs GFS that exist being used by any number of weather services, be that Chris Bedford at Sailing Weather Services in MA; Theyr.net in Iceland/UK, UK MET (own derivation); France METEO (own derivation); Intellicast.com (own derivation); Weatherflow/Sailflow (own derivation) and others. In short, as many flavors as there are Wx forecasters you have their "favorite" models. The key is how data is aggregrated, cleansed, then processed. HOW you process that data in the Wx model is critical to outputs. Some believe in a sole model, others in an ensemble model (e.g. a "blend" of multiple models). There is no perfect solution, only the "best outcomes" solution on a probabilistic basis.

In other words, ClearPoint to date has proven its model works best in real-world scenarios.

Posted by: StuJ24 at November 9, 2007 9:25 AM | Reply

I've read this string but, I don't understand the arguments for a paid weather service.

For example, I use Maxsea Software($600)and the same models are available for free for LIFE if you subscribe to their CHOPPER Server. When I cruise, I simply ask Chopper to send me a compressed daily email which automatically appears in my Outlook Inbox. I click on the file attachment and it automatically updates and links the file to Maxsea everyday! I also use Sailmail and HF Radio with my ICOM710, so I don't pay ANYTHING for airtime. Chopper makes it easy to choose the specific area I need and keep the files to under 3kB which I purposely limit in Outlook. The files are tight and the service is truly AWESOME for those on a budget like myself. I can select the various NOAA and European GRB models that are freely available to anyone on the planet. I've heard that the fancy guys use the Maxsea Sailboat Weather Routing feature to help them win races but, I'm not into that....
I believe that Clearpoint also works but, at the same time, I believe that they are probably just a couple dudes performing the same task as I use with Maxsea and charging for it. Clearpoint doesn't have their own satellite or any secret technology. If they say they do, it's Hooey!

Posted by: Rick_Nav at November 9, 2007 11:05 AM | Reply

Rick, how you know what's hooey and what isn't? You clearly haven't even tried the ClearPoint product---it's in a different league from MaxSea weather, though that's a good service---and I doubt you even checked out their web site.

In fact, I'd put the hooey on youey!

Posted by: Ben at November 9, 2007 1:44 PM | Reply

I'm giving the trial a trial for a race this weekend, right now the model is significantly different from the NOAA forecast in terms of wind strength.

Posted by: bobcongdon at November 9, 2007 2:28 PM | Reply

Rick,

Whoa ... not so fast ... not everyone's products are equally accurate (or equally inaccurate if you take that view of the weather), and it goes beyonds even the models.

I have direct experience with one of the weather services that is having issues delivering the correct current weather conditions as well as the forecasted weather for the Long Island Sound area.

In addition to their being some variation in accuracy in the weather services, the compilation/consolidation, transport, and display of that information to the end user involves many steps that further effect the quality and usefulness of the resulting information an end user receives, and due to the difficulty and cost in comparing actual on the ground weather observations to a products output, the sometimes scant quality assurance efforts leaves a remarkably wide range of quality that one service can use as product differentiation from another.

For example, as methods of customizing forecasts for markets and forecasting models themselves are improved in various ways, I have learned it is remarkably difficult to perform quality assurance measurements on that resulting information, to determine if the improvement is realized at the end of the product delivery by the user. For a specific product I learned about an improvement that uses additional doppler radar information from a specific elevation, to enhance the prediction of rain at ground level and direction of a storm cell. Problem ... wasn't raining at my current position, and although their was rain nearby, their was blue sky above me for 10+ miles in every direction. Whatever QA step was done to evaluate this improvement, wasn't successful, and the resulting product impacted substantially.

I have even found one companies free product to be better than their paid product, as the paid products features had introduced errors in the compilation step that was suppose to add value to the paid product.

So ... I would believe among the multiple vendors competing for our marine weather dollar, some have better products than others for specific uses, and assuming some of the dollars goes to quality assurance, one vendor can remarkably differentiate their product from having better quality vs. being flat out wrong.

Posted by: Dan (b393capt) at November 9, 2007 4:00 PM | Reply

Clearpoint appears to have differentiated themselves in many useful ways.

I have not dived into it too, as I don't see a way for me to use their tool both on land and on my boat (economically). It's difficult enough building some confidence into understanding one products predictions, without having to swap between two products (at sea vs. at home when your planning your trip).

Maybe a mobile phone version of their product could be helpful, I had dropped that idea off in a chat with their technician. I suppose I could use a EvDo card in my PC, but I am not willing to spring for the $100/month cost, nor bring my PC onto the boats I often race with.


Posted by: Dan (b393capt) at November 9, 2007 4:11 PM | Reply

Ben,

Indeed, I have tried the Clearview product and stand by my "Hooey" assessment!!

It appears to be no different from the free Maxsea Chopper Web Server data that I use. I compared the wind data side-by-side as well as some of the other data.

One important point about the Clear Point service is that it sucks a huge amount of data that isn't even required. It is very true that a high speed connection is mandatory. Therefore, it is not a candidate for Sailmail or any other medium speed connection service. At sea, it is simply not an option for me because I won't pay for the data, let alone the subscription.

If Clearpoint is indeed superior and worth the price, exactly what is exactly superior about it, other than the price of the subscription compared to the free Maxsea Chopper Service?

Prove to me that I'm way out of line and I'll gladly retract my words.


Posted by: Rick_Nav at November 9, 2007 7:08 PM | Reply

Aw, Rick, you seem to be blinded by love, or something.

1. ClearPoint doesn't have to use a lot of bandwidth now, and will get improved in this area, as explained several times here. Most boaters have lots of bandwidth when ashore or near shore, but very little beyond a few miles out. Therefore the ideal weather product would be easily usable in both broad and narrow band modes...doncha think?

2. Here's a list of ClearPoint data types. How about you mark the ones offered by MaxSea:

Winds
Waves
Fronts (not just pressure gradients)
Radar (Nexrad, where available, seems to be updated several times an hour)
Satellite (photos, infared)
Observations (buoys and shore stations)
Point Forecasts
Bulletins (seems to have offshore and inshore forecasts and warnings)
Currents
Air Temperature
Sea Temperature
Clouds
Precipitation
Visibility
Sea Ice
Lightning
Tropical Storms
Winds 1 nm or 5 nm resolution (some locations)

Posted by: Ben at November 10, 2007 11:27 AM | Reply

Ben,

To be clear, I am referring to Model Forecasted Weather Information which is in line with the spirit of the previous posts.

Wind, Waves, Current, SST, Altimetry, etc..
In this area, I can see no advantage with Clearpoint. If anything, it seems that Maxsea is better. It's not love, it is appears to me to be factual.

Check out www.navcenter.com

Posted by: Rick_Nav at November 10, 2007 6:29 PM | Reply

Good thread. I've pretty much settled on MaxSea with their full weather package and I'd like to hear from StuJ24 specifically addressing the advantages of his product and what it might offer to a circumnavigator as an addition to their MaxSea package.

Posted by: Merlin Brasil at November 10, 2007 8:17 PM | Reply

I'd also like to hear Stu's reply to Merlin's request.

And to give Clearpoint the benefit of the doubt, let's assume for the moment that their forecast is better. Can I use their data to feed the MaxSea weather routing module?

If not, then I have a horrible choice to make 1) better information processed by a lower quality router (me), or 2) poorer information processed by a higher quality router (MaxSea).

Stu, which is it?

Posted by: Russ at November 13, 2007 8:29 PM | Reply

Hi Ben, et al

This thread never got completed and my questions and Russ's are still outstanding.

Any way Stu can respond here so everybody gets his answer?

The Furuno/MaxSea marriage seems to doing extremely well on their "honeymoon". I'd bet on both of them individually many years back and I'm most pleased to see them thriving together.

Thanks,
Merlin

Posted by: Merlin Brasil at December 1, 2007 4:34 PM | Reply

Lots of interesting stuff here on weather forecasting. Here are my comments, based on a 3 year EU research project in the field, a regular speaker at the Royal Institute of Navigation's weather at sea seminars, and also my company produces the SmartMet weather software.

GFS is not the only global model - others are available from ECMWF, UK Met Office etc - nor is out out of date, but it is the only free one. In fact WDT (www.wdtinc.com), who supply the forecast data to ClearPoint, supply GFS data for the low resolution data. It is also used by Maxsea, UGrib, and most of the other marine weather services - others generally use the US Navy WW3 model, which offers slightly different results, and also has smaller data files as there is no data over the land.

Most higher resolution models take the output of a global model as input, and run a local model which has better topography and better modelling of thermal winds - sea breezes, Mistral, thunderstorms etc. This is what Theyr Iceland do, and also the data providers for Theyr UK (I don't know who they are - possibly WDT?). Some of the models can also accept additional weather observations, but in Europe at least this raw data is not available from the Met Offices at an affordable price, so only high res models from Met Offices are likely to do this.

Our experience is that there needs to be the ability to select what data you want to download, to minimise data volumes. We actually don't use GRIB, as there is quite a lot of repeat data, but send data in our own format, which is more compact. We have also been experimenting with users having the ability to set criteria, so weather data is sent to them when the criteria are fulfilled (e.g. tell me when there are 30kt winds within 100 miles of my position).

Also, different user groups need different data sets. Racing guys want wind, pressure, waves etc, cruisers want cloud and rain, fishermen want sea surface temperature. Also day sailors want current weather reports, whereas liveaboards and ocean sailors want long term forecasts. Services like Clearpoint, ourselves and Maxsea offer a range of products, whilst others offer a more limited choice.

One big difference between paid for and free services is that the reliability of the paid for services tends to be higher, as there is the budget to put the necessary resources in. For example, when I was at Marine Computing Intl we were the leading dealer for Maxsea in the UK for many years, and quite often had frustrated customers because Chopper went down on a Friday and wasn't fixed till after the weekend. Also paid for services have the resources to implement a richer set of data than just the basic GFS or WW3 output. So to some extent you do get what you pay for.

I hope this casts some useful light on weather servies. For more information, you can access Powerpoint presentations on the subject in the Publications part of our web site, www.smartcomsoftware.com.

Posted by: Tim Thornton at August 26, 2009 8:06 AM | Reply

Leave a comment