Polygon Files Structures - Solid Color Maps
Submitted by Ricky Coppola on November 2nd, 2007
Ricky Coppola
Posts: 11
Joined: November, 2007
I'm interested in the use of Polygons. I work a lot with maps and is relatively easy to export from shape (ARCVIEW) to any format. Can you provide me with details of the file structure.
Thanks
Comments
Marc Rueter
Posts: 32
Joined: October, 2007
Instead of explaining it all, I've attached an example. This has the datasource and several examples in a packaged workbook. Note the importance of having all the points in the polygon and the order that those points are supposed to be connected.
Dan Gerena
Posts: 51
Joined: November, 2007
Marc,
I would like to do domething similar, but instead show a map of New York state, with polygon shapes for each zipcode. Can you share what you used as your source of the lat/long numbers so I can recreate on a state level?
Erin Easter
Posts: 520
Joined: October, 2007
Dan,
The Packaged Workbook that Marc attached can be unpackaged to reveal both the data and background images used in the workbook.
To unpackage the file:
1. Right-click the .twbx file in Windows explorer.
2. Select Unpackage.
The resulting folder has a spreadsheet in it called election.xls, which contains an example of how the latitude and longitude values were structured.
However, for convenience, I've attached the MS Excel file below.
Dan Gerena
Posts: 51
Joined: November, 2007
Erin,
I do follow as to how the underlying data forms the shape of each state (and how to observe the underlying data). But what I was hoping to do was to recreate this for a single state (New York), and to setup a file that has the lat/long for the counties withing the state.
Thus my question is not how it was done, but rather the SOURCE of the lat/long points that are used. I'm trying to identify where one would get the lat/long values needed to accomplish what I would like to do.
Dirk Karis
Posts: 42
Joined: January, 2008
One source for Zip code boundary files is the US census, http://www.census.gov/geo/www/cob/zt_metadata.html
These are actually ZCTA (Zip Code Tabulation Area) boundaries, as Zip codes are delivery routes and do not always correspond exactly to a polygonal area. See the Census web site for more discussion of the differences.
The census boundary files may be excessively detailed, causing Tableau to spend a long time rendering them.
Anonymous user
Hi Dan,
Don't know if this is too late, but zip code boundaries and labels are built into the "Map Options". Tableau will do it for you.
Mandy
Anonymous user
Hi Mandy -
Can you give more guidance on how to access those boundaries and load them such that they can be used to create a thematic map?
SCR
James Baker
Posts: 2175
Joined: December, 2007
Zip code boundaries are a layer that can be turned on in the Map Options card (View -> Other Cards -> Map Options if it's been turned off).
While you can map polygon data, you cannot create a thematic map based upon the map layers provided.
Btibert3
Posts: 32
Joined: November, 2008
Hi guys,
I was just about to ask this question, and realized there is a good dicussion going on.
I am a complete newbie to Tableau, but have mapped data in Arcview. What do I have to fill in polygons here in Tableau - right now, I just want to fil in states.
I know you gave a great example, I just need to be explained the process. Essentially, what data files do I need to bring over from Arcview and how do I attach the data in Tableau?
Many thanks for your insight,
Brock
Anonymous user
All,
Great posts. It appears I am missing the step of how to convert the census file format (Arc/info Arcview) into something that I can inport to Tableau (access, xls, etc...) eg. election.xls
Thanks for your help on this.
-Chris
Cush
Posts: 2
Joined: February, 2009
As an update: I was able to hack through the ascii file to pull the information for Georgia and create zip code level detail. Still looking for a quicker way to convert those census files into Access. It was quite tedious and would not be time effective for all 50 states.
Dan Gerena
Posts: 51
Joined: November, 2007
It'd be nice if Tableau could format the files and make them centrally downloadable to each of us, so we don't all independently recreate the wheel. Tracking them down and manipulating is the hard part of this process.
marjenmiller
Posts: 30
Joined: February, 2009
Chris (Cush) - can you explain how you crunched the numbers from ASCII and how you got it into Tableau?
joemako
Posts: 882
Joined: September, 2008
I wrote a macro that rearranges the raw data in the 104 ASCII files, merges them together and puts that data into a normalized Access database. Sometime in the next 12 hours I'll upload the .mdb file.
joemako
Posts: 882
Joined: September, 2008
An Access 2000 .mdb file of the zip code data is being uploaded right now (50 MB) to my server. I'll have the link here in about an hour. It has 52 States with 50,080 Polygons with 5,732,964 Points. The compressed file expands to about 400 MB.
Please be aware that this data is at least 9 years old, and that is is not recommended to have Tableau draw all the zip code polygons for all the states at once.
Sometime this week, I plan on making a short video tutorial on how to use this data.
If you have any isuess or comments you can contact me here or via http://twitter.com/joemako (or if you need help reformatting your data for use in Tableau)
joemako
Posts: 882
Joined: September, 2008
The zip code data can be downloaded from http://joemako.com/files/ZipData.rar
This is a .rar file not .zip, you will need a application like WinRAR 3.80 that can be downloaded from http://www.rarlab.com/download.htm
Btibert3
Posts: 32
Joined: November, 2008
Hi Joe,
I know am asking alot considering you already uploaded a database for us to use, but have you had a chance to create a brief tutorial for newbies like me on how to associate data with the polygons?
Thanks again!
Brock
marjenmiller
Posts: 30
Joined: February, 2009
Good idea Brock. I could use it too.
marjenmiller
Posts: 30
Joined: February, 2009
Good idea Brock. I could use a mini write up too on how to use the data. Thanks!
joemako
Posts: 882
Joined: September, 2008
I spent some time this weekend working with the data, trying to find a meaningful way to use the zip code data from 2000, and was unable to.
I did some more searching and found this thread, http://www.reddit.com/r/programming/comments/7z8al/where_can_one_get_zip... where people agreed that the 2000 data is not useful. A good read if you are thinking about using zip code boundaries.
The thread did point to http://www.census.gov/geo/www/tiger/ that has data that looks to be both more current and more useful.
I will take a look at using SharpMap, http://www.codeplex.com/SharpMap to extract the data from the ESRI Shapefiles to make it usable in Tableau.
I really wanted zip boundaries to work, but the more I learned about them, the more I realized that the way Tableau maps zips is more effective on every front than anything I could put together with zip boundary data.
I am going to take a look at compiling state boundary data, dealing with islands in states, and work on putting together an easy process for creating custom region boundary data.
Sorry no video yet :( Marc Rueter did a really good job, and I am still learning off of his work. When I can create a map like that from scratch, then I will have video.
Thank you for your patience.
Anonymous user
Can anybody point me in the direction of some more detailed training on how to establish a set of boundary overlays on a map? I am looking to establish some mapping visualizations that depict school district and school boundaries and where students live in relation to the schools' boundaries.
Thanks,
Danny Combs
joemako
Posts: 882
Joined: September, 2008
Danny Combs,
I would be glad to help out, I have some free time this upcoming week. Feel free to contact me via email joemako [at] gmail [dot] com or on twitter http://twitter.com/joemako.
sbenners
Posts: 6
Joined: April, 2009
Wow, there is obviously a lot of buzz going on in this area. I'm new to using Tableau and I'm currently doing an R&D engagement with our data. I looked at the elections.xls file in the packaged workbook example provided towards the top of this thread. It has two fields in it that I'm interested in: PolygonId and PointOfOrder. These two fields are obviously critical to creating the Polygon Map example which is very similar to what I'm looking to create. I don't have this data and I'm interested in obtaining it. Currently I've got city, state, zip code, and lat/long coordinates...how do I go about obtaining these last two fields though? Anybody know?
joemako
Posts: 882
Joined: September, 2008
sbenners,
PolygonId is a unique identifier number for a polygon, most states have only one, but some states may have more, because of islands.
PointOfOrder is a unique identifier number for each point/vertex of a polygon sorted in order to drawn, more complex states will have more. Each PointOfOrder will have a Lat/Long coordinate.
If you would like to discuss more details/help on creating these visualizations yourself, contact me on twitter or via email (see above).
sbenners
Posts: 6
Joined: April, 2009
Within the elections data there are 5 states ('NY','MN','VA','MI','TX') that have more than one Polygon Id associated with them. In my smaller subset of data I'm trying to determine which Id to use with each Latitude and Longitude coordinate since my set of coordinates are different from the ones provided in the elections data. It feels like it is a guessing game (i.e. numbers just being close) rather than a science. I wish the technical support at Tableau would provide more insight and direction in this area since the Polygon File Structures require both the coordinates as well as the Polygon Id and Point of Order.
sbenners
Posts: 6
Joined: April, 2009
Well, I went through the steps to populate both the PolygonId and PointOfOrder fields. It looked good when I finished it. However, when I applied my source data to a map, it looked like crap. Polygon structures were being painted all over the place and it looked like the work of a little kid in art class. I sure wish someone from Tableau could provide further insight on the process of populating these fields, their validation, and the science behind applying them. I know it's something that I'm doing wrong, and I'm sure I could get things cleared with a little assistance from the creators of the software.
joemako
Posts: 882
Joined: September, 2008
OK, let’s see if I can approach this in another way. Instead of looking at states, let’s take a look at some simple shapes, and relate it to working with states.
Notes on the attached image "layout.png":
- Shapes A and B are single contiguous polygons (Similar to a state like Utah)
- Shape C has two polygons (Similar to a state with an island, like Texas)
- Points are ordered like a connect the dots game, if you change the order, the shape will be different.
- For each polygon, the last point is a repeat of the first point to close the loop (this may not be required, but I think it is a good practice)
- The values on the X and Y line can be used similar to Lat/Long
Notes on the attached Excel file "shapes.xls":
- On the "Shapes" tab, you can see 3 different Shapes (A,B.C), 4 PolygonIDs (1,2,3,4), 19 PointOrders (1-19), and X and Y coordinates for each point. All this data came from the layout.png image. (similar to state polygon/point data in that states are like shapes)
- On the "Data" tab, this would be your source data, Shape A has 1 store located in it, Shape B has 2 stores, and Shape C has 3 stores. I've also included sales figures for each store.
- The "Combined" tab is what you get after merging these two data sets together, (I'll get into how it was made next).
On the "Shapes" tab, 5 rows describe Shape A, 6 describe B, and 8 describe C.
Taking a look at the "Combined" tab, you see 41 rows of data, 5 for Shape A, 12 for B, and 24 for C.
So 5 rows became 5, 6 rows became 12, and 8 rows became 24. The math is (5*1)+(6*2)+(8*3) = 5+12+24 = 41 The 1,2,3 came from the number of stores in the corresponding shape.
Attached is an image "query.png" that shows the two tabs and the query that generated the "Combined" tab.
The query is simply a query with both tables, joined on Shape that include the * from Shapes and the fields you want from your data.
When we have Tableau connect to the Combined data source, we are able to see the result, attached as "Shapes.twbx".
I made a couple of tabs that provide different views of the three tabs.
If any of this does not make sense, please let me know what part(s) I was not clear on.
I have also attached a "States.xls" that came from the elections.xls file that has the same structure as the "Shapes" tab (State abbr. instead of Shape, and Lat/Long instead of X/Y)
Best Regards,
Joe Mako
joemako
Posts: 882
Joined: September, 2008
I just want to make sure my stance on this type of data visualization, using state polygons, is clear.
I do not think this is the best way to do this.
You can see some examples on http://flowingdata.com/2009/05/04/10-visualizations-for-number-of-days-t... where bar graphs with Tableau are easier to gain understanding of the data than a state polygon map.
Sometimes it does work out well at the county level, see http://www.nytimes.com/interactive/2009/03/10/us/20090310-immigration-ex... as most counties are a similar size in area.
Stephen Few clearly states best practices when geocoding data at http://www.perceptualedge.com/articles/visual_business_intelligence/geog... that I strongly agree with.
As always, just because you can do something, does not mean you should, and if it is worth doing, it is worth doing right.
Other than that, I do see a future use for polygons in Tableau, but not with state shapes. I have not fully formed or tried my ideas out yet, but when I do, and if they are useful, I will start another thread.
Anonymous user
I'm a newbie to Tableau too. I'm looking to synchronize Tableau with ESRI products and was wondering if it was possible to add in my own polygons that I create, not just the predefined States, zip codes, etc. from Tableau. Do I have to connect to an server with ESRI supported data?
Anonymous user
I am trying to build a densities map, similar to what I would normally do in ArcGIS. I am having some difficulty since I am trying to do it at the zip3 or SCF level. Any idea where I can find a polygon for this?
haniar
Posts: 12
Joined: July, 2009
Hi All,
Tableau geo data is very much geared towards US. Does anyone know where I can get country polygones, or even internation cities such as London lat/lon coordinates.
Julius
Posts: 85
Joined: May, 2009
Hello all, just wanted to see how Tableau is doing with any new capabilities related to choropleth or thematic mapping. I've seen the point density maps as well as examples of shape drawings but was looking to see if there are more intuitive ways to create something similar to the attached image (county_choropleth.gif)
Also would like to plug a couple of blogs related to mashup efforts around GeoBI:
http://blog.thematicmapping.org/
http://highearthorbit.com/geocommons-cloudmade-integration/
Thanks.
[UPDATE]: Sorry for being redundant, Joe. I just realized that you had a similar posting on 5/4/09 regarding the use of choropleth maps in NYTimes. That's exactly what I was inquiring about and wanted to see when we can expect this feature. The bubble map is a great start but with dense amount of data bubbles start to overlap and lose detail - would love the thematic option. Thanks!
Anonymous user
Stephen Few explictly endorses thematic maps/choropleth maps!
"Two approaches to displaying quantitative information on maps usually work best: variations in color intensity, in size, or both. The map below, which I borrowed from Gretchen Peterson’s excellent new book GIS Cartography: A Guide to Effective Map Design, illustrates the use of color intensity for displaying quantities. Imagine that this is a country divided into provinces, and that various intensities of the color orange are being used to encode average household income—the darker the color the greater the income. This approach displays an aggregate measure for each province rather than a measure for each household. Geographical displays of this type are called choropleth maps."
Why can Tableau not:
1. Let us apply our own "data layers" to maps as you do with population, et al.?
2. View county borders at a national view?
I sense that what you are saying is that this would be really hard to accomplish in Tableau's current architecture. That is not a reason not to do it. I'd be happy with choropleth maps separate and apart from Tableau's main mapping construct. No need to worry about drawing marks on top of thematic maps. Just let me have the option of thematic maps at the state, county, and MSA level for the U.S. and keep it simple for me and yourselves. Give us what we want and what our users expect, in other words.
Anonymous user
Any update on this. This feature would be really useful to roll out across all 50 states.
Anonymous user
Does anyone know a way to have the Polygon Id (or state name) show on the map so people can tell the names of the states.
Anonymous user
hello!
hey people please do not sleep and let us know how polygons of shape files regarding non us countries can be
included in your programm. you have such fantastic software product which also we in europe would like to use.
please prevent me to make a visit in your office with the rest of my crew - see photograh below (irian jaya 1998)
Chris Cammack
Posts: 15
Joined: December, 2009
Has Tableau decided to address this yet? It is frustrating to have such a powerful program be limited by a seemingly fixable problem? The anon user from the 25th is right on the money.
Douglas
Posts: 18
Joined: January, 2010
Does anyone know of a source for US county polygons? Or of a way to convert this KML into Tableau-consumable data? http://www.gelib.com/us-counties.htm
Thanks,
Douglas
(add my vote to the request for full chloropleth support in Tableau!)
Anonymous user
was there any response to the posting by Douglas? i would like to get the same data - US county polygons. Please advise if there has been any updates on this.
Douglas
Posts: 18
Joined: January, 2010
Here's the data in SVG format. http://upload.wikimedia.org/wikipedia/commons/5/5f/USA_Counties_with_FIP...
Would be lovely if someone would convert one of these to Tableau-happy format.
Anonymous user
Douglas,
That is a great SVG file, but none of the coordinates are lat/long, they are pixel x/y. If you do not want to use lat long, then it might be useful. Is that what you are asking? Even if the boundry shapes are extracted into a CSV, you will still need to process the data (see joemako's comment above from April 30th, 2009).
You may be looking for the Shapefiles from http://www2.census.gov/cgi-bin/shapefiles2009/national-files
Anyone have a resource for converting Shapefiles to CSV?
Be aware that polygon files are not very powerful in Tableau, nor are they quickly rendered in large scale. You may be better off setting up a WMS server, and use data layers.
Another better choice is use a bunch of marks like in the video at http://www.tableausoftware.com/videos/advanced-mapping-techniques starting around 33:00, but that approach is only available against a data source like SQL Server because the RAWSQL function uses geom.
Anonymous user
OK, as kind of a newbie here I have pieced together what needs to be done to use polygons for thematic maps based roughly on zip codes (understanding that zip code polygons roughly approximate delivery areas).
I downloaded the latest zip code shape files from here
http://www.census.gov/geo/www/tiger/tgrshp2009/documentation.html
and I used SAS to extract the raw data fdrom the shape files.
However, I don't see anything that seems to corresponds to the "order" or "point order" field that indicates the order in which the polygon line segments are connected.
Everything else seems to be there.
Anyone know how I should decipher or interpret the data from the shape file?
Variables extracted, with sample data, are
X, Y, SEGMENT, ALAND, AWATER, CLASSFP, FUNCSTAT, INTPTLAT, INTPTLON, MTFCC, ZCTE5CE
-112.617833, 44.241085, 1, 1667073388, 272487, B5, S, +44.2171357, -112.4327143, G6350, 83423
How do I get and "ORDER" out of this?
Thanks.
stud_student
Posts: 5
Joined: March, 2010
BUMP.
I'm trying to do the same thing - figure out / extract the point orders from the census data.
joemako,
Excellent post explaining how point orders work.
Post new comment