Liberating your My Maps data

Richard in his post at the OpenGeoData blog, highlights the work of Google’s Data Liberation Front which aims to make sure that user data hosted on Google servers can always be exported out for use in other services or applications.

So what of Geodata, well contrary to popular opinion if you create a My Maps mash up you are just one click away from exporting your map data as a KML file;  just click on the link marked “View in Google Earth” and a KML file of your map is downloaded.

Richard asks if it is possible for Google to offer a “mass tracing” right similar to that offered to the Open Street Map Community by Yahoo. This I’m afraid Google cannot currently do as we don’t have the rights to offer this on a universal basis.

I hope this is a useful clarification, sorry I could not add a comment on the blog itself for some reason.

Written and submitted from home, using my home 802.11 network

49 comments

  1. John

    Your blog post was a little vague, does this mean that points that were pin pointed using Google maps/streetview/sat imagery as long as it isn’t vector data isn’t restricted in re-use?

  2. Ed

    Hi John,

    You can create vector data from Google Maps, via My Maps for example or a Maps API application, but this must not be for the bulk creation of data. From the terms of service..

    “Also, you may not use Google Maps in a manner which gives you or any other person access to mass downloads or bulk feeds of numerical latitude and longitude coordinates.”

    ed

  3. John

    I was talking about points, it was clear from your post that vector data isn’t allowed but things like http://www.wikitude.me generate POIs from gMaps etc.

    Is this sort of activity specifically an unrestricted allowed use or are they also in breach of google T&Cs?

  4. Ed

    @John, Points are vector data ?

    @Ivan MapMaker is different, Google has the appropriate rights to use imagery for this application in particular areas.

  5. Frankie Roberto

    So it all comes down to the definition of “mass downloads or bulk feeds”.

    I guess this means that me tracing my 6 hour walk and exporting that (I do actually do this), and then re-licensing it as a walking route is ok.

    But systematically tracing every single road and re-licensing that (in the way that OSM contributors might do) is out.

    Still a bit vague though – what if a large number of people individually trace their walks and share them, and then someone aggregates them all together to produce a map?

    And is there a difference between tracing the map tiles vs the satellite tiles? What about if I upload my GPS trace to My Maps, and then adjust it slightly using the satellite photography (eg to improve bits where the signal was weak). Does that add any restrictions?

    It’s decidedly murky.

    Can’t Google just acquire the satellite photography companies that you’ve licences from? :-)

  6. Richard Fairhurst

    (A bit easier to reply here than with 140 Twitter characters!)

    If you make a document with Google Docs, then decide you want to use it elsewhere – say, upload it to Adobe Buzzword, or print it out and give copies to your mates, or whatever – you can do that. There’s an export function and there’s no restrictions on how you use your exported data.

    If you organise your life with Google Calendar, then figure you’d rather use iCal or, perhaps, Outlook on your work intranet, again, that’s no problem. Google doesn’t restrict what you do with the data you created.

    But if you create a mashup with Google Maps… yes, you can get the data out as KML. But unlike the others, you’re restricted in how you can use it. You probably can’t upload it elsewhere. You probably can’t publish it. You can view it on your own computer with Google Earth, but that’s about it.

    So that’s the Data Liberation angle.

    Whether or not your current contracts allow you to offer this – well, that’s your business. :) I’m sure the Data Liberation guys are willing to get stuck into legal issues as well as technical ones; certainly YouTube, Google News etc. have shown that Google is never shy of tackling ‘content providers’ about copyright for users’ benefit.

    (I’m always a little bemused by the suggestion that Yahoo buys more rights than Google does. If I were Yahoo, would I spend an extra £10,000 on some obscure rights, or on aerial imagery of Birmingham? But you’ll have more of an inside track on this than I do.)

    What we really need is clarity. That’s why the Moderator request just says “sort the legalities”. Right now Geonames, for example, lets you mash up vector data from a Google aerial image and export it elsewhere (as a CSV, say) whereas OpenStreetMap doesn’t. There’s several more examples where Google allows sites like this to persist. My reading of the Ts&Cs is that you don’t actually forbid it.

    A real Data Liberation success would be to make Google Maps mashups as unencumbered as anything else on Google. At the very least, a definitive, unambiguous statement of what users can and can’t do is long overdue.

  7. John

    Ed can you please be more specific, is wikitude.me is in breach of Google T&Cs or not?

    Wikitude allows bulk downloading of data created using google maps, sat imagery or street view depending on what their site implements.

  8. Ed

    @Richard,

    Indeed the limitations of real time micro blogging :-)

    About Mash-ups I fundamentally disagree Google makes no claims over the ownership of your data (unlike some others) and you are free to use KML data in whatever way you wish, not just in Google properties.

    I can’t comment of Yahoo’s rational or approach to licensing data, or I can do is try to explain why Open Street Map is currently unable to use imagery from within Google sites in the same way.

  9. Ed

    @Ivan,

    Ok for MapMaker data is is still early days, but this page explains Google’s approach, and allows you to download data for personal or non-commercial use.

  10. Ed

    John,

    I don’t know the details but does wikitude,me allow people to export all the users contributions for use outside of their AR application, which i Love BTW !!

  11. John

    Ed, I was under the impression that wikitude.me releases their data under CC-by-SA, perhaps it was another similar service, but there is at least one service that does do this.

    Back to wikitude for a second, what does bulk download mean exactly, 100 points at a time, 1000 points at a time, 1,000,000 points at a time?

    Also why is there a non-commercial clause on the mapmaker data, google is obviously using it for commercial purposes or plans to, if I contribute to mapmaker why can’t I take my contributions and do what I like with them?

  12. Richard Treves

    Apart from the discussion about rights I think there is a more direct issue for the Data Liberation guys in how data is exported from Google Maps. It comes out as a network link which means that the actual data cannot be saved onto a local drive unless:
    - you’re prepared to tediously copy KML features one by one to the places column
    - you’re techie enough to know how to grab the kml from the network link

    An option allowing export as non network link as well as a network link would be much more open and, if I remember correctly, Google Maps used to only offer a none network link export.

  13. Ed

    Richard,

    Ok I accept the process could be a lot simpler, but if you are geek enough to want to to export your data to display it in a GIS application this would not be beyond you.

    You would prefer direct to KML and shp ?

  14. Richard Fairhurst

    At one end of the spectrum you have “Google makes no claims over the ownership of your data”. At the other you have an absolute prohibition on “mass downloads or bulk feeds”.

    So there’s a dividing line somewhere, between ‘standard’ mash-ups and mass downloads. The problem is that Google has never said where this line is.

    This actually isn’t an OpenStreetMap-specific question. When I referred to the SoC Summer School in the OpenGeoData posting, I was attending the event for my employer (WW Magazines), not as an OSM contributor.

    One of our magazines is a 4×4 off-road driving magazine, and without giving too much away, we’re thinking of doing some off-road webmapping with a small crowdsourcing element: click on this ford or this incline to rate it, that sort of thing. Should work brilliantly with aerial photography.

    But I can’t recommend to our company that we do it with the Google Maps API, because there is so little clarity about what’s allowed and where the IP will lie. It might be ok with 50 fords. But 100? 1000? What happens if I want to take it out of Google Maps entirely, and mash it up with OSM vector data (and, ahem, here there may be some connection to the live renderer I’ve been working on)? OSM’s licence is ok with that (it’s a Collective Work), but I honestly couldn’t tell you if Google’s is. What if we want to use the information in the printed ‘roadbooks’ we feature in the magazine? We can’t print 10,000 magazines then pulp them due to a copyright infringement.

    There is, as yet, no answer on this. That’s why the Moderator request says “Sort the legalities” – not simply “Give OpenStreetMap permission”.

    That said:

    I do think that you will do much, much better by opening it up fully, and no doubt so do the (quickly checks) 992 others who’ve voted for the request. Yahoo have benefited directly from the OSM arrangement, with a whole host of good, free maps for Flickr. Most OSMers I know would love Google to use some OSM data, and ODbL means that yes, we are getting our legal act in order. It would be great if you were to do the same!

  15. Ed

    Richard,

    Thanks for your thoughts, I appreciate your position but I don’t agree with a number of points you put forward.

    Google makes no claim over the intellectual property contained within user contributions regardless of the size of those contributions from one point to a million.

    There is at this time a restriction preventing people using Google Maps for bulk data capture as would be required for a project like OpenStreetMap. This would not prevent the application you mention or many of the other large projects which currently create new data.

    There is a clear difference between collecting a few hundred km’s of green lanes or the locations of every Tescos store and capturing the 350,000 km of the UK road network for use outside of any Google property.

    I really don’t see this as an issue of legal clarification, rather you don’t like the limitations of the current terms of service, specifically to be able to use the imagery licensed by Google to create large geospatial databases without restriction.

    I’m really sorry this is not possible at the moment..

  16. John

    Ed you seem to be dodging our questions rather than answering them.

    If Richard uses a Google map API to collect 1000000s of points and then decides to place these markers on printed maps using OSM tiles would this be an allowed use under current Google T&Cs?

    Could he also print out the lat/lon of those points in his magazine as well?

    And could you please answer my question about MapMaker, I’m not asking about the complete data set, just if I made any personal submissions.

  17. Ed

    John,

    No that sounds like bulk export of points to me.

    Re MapMaker as I said in the original answer, we are just rolling out the export of mapping data from MapMaker and currently we have implemented export of the complete dataset for non-commerical or, personal use only ..

  18. Barry Hunter

    One clarification on the “View in Google Earth” link – if you use that and expect the file to contain the actual data of the My Map – it doesn’t. It’s simply a link to tell Google Earth where to download the data from the GMaps server (designed so GEarth picks up changes to the My Map automatically – a useful feature in itself)

    How to get a KML file containing the actual data:
    http://sites.google.com/site/gmapstips/export-my-map-as-kml

  19. Tim Woolford

    In response to Richard’s post. Rather than ‘deriving’ co-ordinates from a map, having returned from greenlaning, the other approach is to take a geotagged pictue of the relevant feature while you are actually there. The lat/long will then be embedded in the EXIF data of the image (Ed – you know how I’ve been pushing geotagged images for navigation). Given most (all!) green lanes are in open countryside the gps fix will be very good. The resulting images can then be uploaded to Panoramio, Flickr, NavPix etc. This gets over all the derived data issues as you have captured the co-ordinates at the location. The other big advantage of the approach is the actual condition of the track, feature etc is recorded which I know is a big issue.

  20. Ed

    Tim,

    Nice idea, although as I have pointed out there would be no issue in derived data from that application.

    I guess using GPS would be just like the old days for many OpenStreetMappers :-)

  21. SteveC

    Ed

    Your statements about it not being possible under the current license arrangements Google has with it’s aerial imagery suppliers are well founded of course.

    However, Google is a fairly innovative company and has large purchasing power. If Microsoft can buy Vexcel or some unknown company (wink wink) can buy all the rights to ANDs EU data… surely you guys can figure something out to get people access to this stuff?

    You’re in the business of ripping the biz models from under people, why not the antiquated aerial imagery industry? It’s not like anyone thinks it’s cool or efficient. I vote Google make some (more) moves to drag them kicking and screaming in to the century of the fruitbat.

    And, it’s not like Google have to release every bit of imagery right now in to the Public Domain. Why not start small, just do a area like San Francisco, or how about aerial imagery from 5 years ago in London? Google could say to it’s suppliers, we’d buy 2010 imagery but we want to offer 2005 imagery under better rights, and make that a sweetener on the deal for an extra 5% or something.

    Basically “we don’t have the rights” doesn’t have to be the end of the conversation?

    Your helpful business development advisor,

    Steve

  22. TG

    Reading through this thread I feel I need to make a stand for the aerial suppliers (I’m not one of them).

    If Google were to buy & grant the rights to aerials as some people suggest, I fear the excitment would be short-lived, even if you reserve up-to-date imagery as a high-end niche market.

    The aerial industry is already suffering from tight margins and can’t be doing much better than just about breaking even. For them to offer their imagery to Google et al was already a great leap. It was no doubt lucrative for the suppliers in the short term but they must have been lying awake at night worried about substitution risk. Still, the whole industry and the public have benefited, which is great.

    One more such leap however, giving away virtually all rights, would commoditise aerials even more and possibly result in suppliers falling off the cliff altogether. Unless we are happy to stick with 2009 imagery forever, Google and Microsoft would then have to step in themselves and take over aerial capture, monopolising the market. I’m not sure that would benefit anyone in the long-term – history shows that market choice and competition is always better than the reverse.

    The alternative is of course to just “do it” as some people suggest and see what happens, and new opportunities will always work out. This is often true and seems to be the businsess development model favoured by the 2.0 world.

    But sometimes I wonder whether the costly realities of buying/hiring aeroplanes, fuelling and maintaining them, and having qualified survey pilots flying them, are sometimes overlooked in the debate.

    Your friendly paleo reseller, Thierry

  23. Richard Fairhurst

    Ed – when you say “I really don’t see this as an issue of legal clarification”, I know you personally don’t! You’ve said that before.

    Nonetheless the customer is king and this customer (with, if you’ll excuse the immodesty, more exposure to spatial law than most) doesn’t see any clarity here. I hope that Google will be able to give a more definitive answer on a site which doesn’t have a “not the opinions of Google” disclaimer on it. I’m sure the Data Liberation Front will be working through their Moderator suggestions in due course and I look forward to hearing from them when they do.

    Individual comments:

    Tim – sure, I can do that because I have a camera and a GPS, but until everyone has iPhones it’s not a valid model for a consumer website. Ed – hey, many of us still only do the GPS thing; tracing from aerials for “fun” has never actually appealed to me when I could go out on my bike! And Steve is spot on here and, well, gets it. Please cut that previous sentence out and frame it – as he’ll probably tell you I don’t say that very often. ;)

  24. Ed

    @Steve,

    Thanks for the Business Development ideas, if we are able to grant the rights you need, don’t worry you will be the first to hear. You could also try this two additional ideas..

    - Ask Yahoo to buy more up to date imagery, as they appear to have the rights to grant you what you need.
    - Cloudmade could buy the imagery with the appropriate rights for use by the OSM Community

    As TG points out capturing imagery for millions of square kilometres is not an easy job and it does not come cheap, it will according to the analysts be worth a billion dollars next year, even for Google that is a lot of money :-)

  25. Ed

    @Richard

    OK to be really clear – I think the terms of service are quite definitive..

    “Also, you may not use Google Maps in a manner which gives you or any other person access to mass downloads or bulk feeds of numerical latitude and longitude coordinates.”

    As for the data liberation point, I don’t really think what you are asking to do fits with the general understanding of exporting users data.

  26. Richard Fairhurst

    Ok, my last comment on this post as I think we’re increasingly repeating ourselves. Let’s forget OSM exists for the duration of this comment as it’s just clouding the issue.

    There are two places where that sentence isn’t clear:

    a) It does not define “mass” or “bulk” (my point above)
    b) The Google Maps Terms applies it to ‘Map Information’ and the Maps API Terms to ‘Content’. UK and US case law is clear that derivations from aerial photography are not ‘Content’ or part of the ‘Map’

    Google (on an official site) could do the minimum required to answer the request by:

    1) Answering a) by publishing the quantitative/qualitative criteria in use
    2) Answering b) by confirming whether or not the terms apply to derivations from aerial photography
    3) Listing any further restrictions that apply to data created in this way

    Anything over and above this would no doubt be warmly welcomed. As your issues with the OS demonstrate, and the comments over on James Fee’s blog by trad GIS guys confirm, derived geodata is a live issue for many people.

    I’m sure the Data Liberation Front expect some resistance from product teams. There wouldn’t be much point to it if they didn’t. It’s to be applauded that Google have created an organisation to challenge this resistance, and I look forward to hearing from them.

  27. Ed

    Richard,

    No we can’t remove OSM from this discussion,; it is you motivation to trace imagery for large scale production of vector data that you want to make available to third parties. As I said for most other applications this is not an issue.

    What would you think bulk means ?, it is not uncommon not to put a precise value against the term, the doctrine of fair use is based around the insubstantial reproduction of copyrighted works, but what does insubstantial mean ?

    This point is in relation to the service not any particular type of data.

  28. TimW

    I think “Fair use” is one of the worst terms to ever get written in to contracts. I wonder if it was infiltrated into society by contract lawayers to generate business for themselves as its ambiguity and openness to interpretation is huge ;).

    Mobile phone and broadband contracts are often sold as “unlimited” with a fair usage policy (so not in fact unlimited!). The fair usage policy seems to be a threshold (e.g. downloading more than 10Gb a month) after which the customer is warned, advised to changed to a different tariff, or perhaps cut-off. It’s handy if the threshold/limit is published so you can see what the “unlimited” deal actually is.

    Google must have a threshold set in the system somewhere that triggers at least an investigation into a user’s activities, even if it doesn’t do anything else automatically. What is this “alert” threshold? How many times can you download more than the threshold? Does a user get let off with a warning the first time? Etc etc…

    I like the idea of fair use policies but in practice I think they cause more trouble than they’re worth. Everybody’s definition of “fair” is different for one thing. Is it OK if I decide that bulk means 2 billion plus records, and that what I’m doing is fair? If Google decides what’s fair, what rules does Google use? Or is it just made up on a case by case basis?

    Part of the problem does lie with the particular data in question. A dataset of capital cities will be much smaller than a dataset of street lights. Would downloading the entire capital city dataset (or 90%?, 80%?) count as a bulk download? Say there were 1000 capital cities and you downloaded 1000 street light records from a dataset containing a million records, does this count as a bulk download?

    If Google is not prepared to put actual figures on things, perhps would help if they published some case study examples on what is and is not fair use.

  29. John

    Several telco’s in Australia were taken to court by the ACCC over the usage of “unlimited” and lost.

    As a direct result “unlimited*” has all but disappeared, only things that are truly unlimited can be deemed unlimited.

    However I didn’t get an answer to the same question, is 100 points a bulk download, is 1000, 1000000 etc. What does bulk downloading mean exactly, because you download via xml etc a lot of points on a lot of sites browsing them and having points appear on a map.

    What about the case of kml files which are entirely downloaded to be viewed in an offline KML browser?

  30. John

    Ed I’ve been pondering over your comment how this is about OSM and that just isn’t true and in fact you are skewing things out of context, at least on some points.

    One of the points of contention is if you use a gMaps API to collect information you then seem to be always locked into using gMaps API to display or show the information that was collected, otherwise you may fall foul of the derived information/bulk downloading issue.

    wikitude.me allows a large number of points to be downloaded at a time through their AR viewer, is this bulk downloading?
    If it isn’t, what exactly is bulk downloading?

  31. Ed

    John,

    I think that is the key point, an application like wikitude is not allowing the bulk export i.e. downloading of data. The points are only accessible via their api within an application and then only a relatively small number of point.

    An no you are free to collect points via the goggle maps api, there is not an issue with derived rights, the information collected is your IP.

  32. John

    Ed, but if I was collecting the information via gMpas it isn’t entirely my IP because I can’t just do what ever I like with it, I can’t publish an entire list in a magazine I can’t offer a KML file with all the points I can only serve a partial list online it would seem.

  33. Ed

    John,

    There is a difference between the IPR in data you create and the restrictions described by the Terms of Service (ToS).

    The data you create is always yours, however there is a restriction in the ToS that prevent people using the service to extract large amounts of data for third party use.

    Imagine going the science museum and talking photos of some of the exhibits, nobody would have an issue with this and the photos belong to you. The science museum would I imagine have a problem if you came along with a tripod and lights, and systematically photographed every exhibit over a period of weeks and then held a exhibition of your pictures in Kensington Gardens.

    I would see no problem in KML files that describe specific walking rails, greenlanes etc made available to readers of a magazine, however creating a map of every road in the UK is very different.

  34. John

    Ed, we’re not talking about making maps of every road it’s blatently obvious that this is not an acceptable use.

    What we are talking about is points of interest, like wikitude.me collects, and how that information can be redistributed.

  35. Ed

    John,

    I thought we had established here that what wikitude are doing is acceptable, so if you want to do something similar then that’s fine.

  36. John

    Ed, I was trying to make an analogy over what they are doing, but the question comes back to what is bulk downloading, if they offered a KML file of all their locations is that ok or not?

    Basically where is the line between bulk downloading and not bulk downloading?

  37. Ed

    That would be up to them, as it’s their data :-)

    No there is not a defined line above which extracting data is regarded as bulk. But you can infer that all the current apps using Google Maps are doing so within the terms of service, there is therefore a benchmark of sorts established.

  38. Ian Dees

    Ed,

    From your description, if I use Google’s tiles and/or StreetView to collect data (let’s say by tracing the roads with satellite and naming my traces with the street layer) for all of the streets in my neighborhood, then post that somewhere online with a CC-BY-SA license. It’s my data, right?

    Afterwords, someone in the city next to me does a Google search for my neighborhood, finds the data file, and adds their streets to the data file. Repeat ad infinitum.

    Is that systematic (and thus against the ToU for Google)? At what point did it become systematic and not just regular people collecting their own data? When 3 people collected data? When the data was posted for public consumption? When they started building up a large data file?

    (By the way, thanks for spending on this conversation. I think it’s at least an interesting topic, if not very important.)

  39. Guillermo Sansovic

    Ed,

    The explanation is still a bit confusing to me.

    I have a simple example that will help clarify the ideas involved.

    There is a site in Argentina that collects POIs. It’s very simple: you click on a Google map or satellite image (using the Google Maps api, of course) and create a new POI, which you then tag.

    The very home page of this site explains that all POIs are available under a Creative Commons Attribution-ShareAlike 2.5 license.

    There is also a way to download all the POIs in the site.

    I understand by your comments that they own these POIs which they are licensing under a license compatible with OSM.

    Is bulk uploading all these POIs to OSM a violation of Google ToS?

  40. Ed

    Guillermo,

    Without seeing the site it impossible to be definitive, the issue is with the bulk export of points and as others have pointed out it is not easy to define bulk in any universal sense, a thousand points nobody would have issue with, millions of points would be clearly “bulk”

    My guess is the site you mention is somewhere between these two extremes..

  41. Pingback: Planet Street Map » Google bids fare-thee-well to TA in the US of A
  42. Pingback: Open Mobile Map » Google bids fare-thee-well to TA in the US of A
  43. Pingback: Google bids fare-thee-well to TA in the US of A | J2ME GPS

Post a comment

You may use the following HTML:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>