Polygon Files Structures - Solid Color Maps

Ricky Coppola

Posts: 11

Joined: November, 2007

I'm interested in the use of Polygons. I work a lot with maps and is relatively easy to export from shape (ARCVIEW) to any format. Can you provide me with details of the file structure.
Thanks

Comments

Marc Rueter

Posts: 32

Joined: October, 2007

Instead of explaining it all, I've attached an example. This has the datasource and several examples in a packaged workbook. Note the importance of having all the points in the polygon and the order that those points are supposed to be connected.

AttachmentSize
polygon map.twbx815.51 KB

Dan Gerena

Posts: 51

Joined: November, 2007

Marc,
I would like to do domething similar, but instead show a map of New York state, with polygon shapes for each zipcode. Can you share what you used as your source of the lat/long numbers so I can recreate on a state level?

Erin Easter

Posts: 520

Joined: October, 2007

Dan,

The Packaged Workbook that Marc attached can be unpackaged to reveal both the data and background images used in the workbook.

To unpackage the file:

1. Right-click the .twbx file in Windows explorer.
2. Select Unpackage.

The resulting folder has a spreadsheet in it called election.xls, which contains an example of how the latitude and longitude values were structured.

However, for convenience, I've attached the MS Excel file below.

AttachmentSize
election.xls3.92 MB

Dan Gerena

Posts: 51

Joined: November, 2007

Erin,
I do follow as to how the underlying data forms the shape of each state (and how to observe the underlying data). But what I was hoping to do was to recreate this for a single state (New York), and to setup a file that has the lat/long for the counties withing the state.

Thus my question is not how it was done, but rather the SOURCE of the lat/long points that are used. I'm trying to identify where one would get the lat/long values needed to accomplish what I would like to do.

Dirk Karis

Posts: 42

Joined: January, 2008

One source for Zip code boundary files is the US census, http://www.census.gov/geo/www/cob/zt_metadata.html

These are actually ZCTA (Zip Code Tabulation Area) boundaries, as Zip codes are delivery routes and do not always correspond exactly to a polygonal area. See the Census web site for more discussion of the differences.

The census boundary files may be excessively detailed, causing Tableau to spend a long time rendering them.

Anonymous user

Hi Dan,

Don't know if this is too late, but zip code boundaries and labels are built into the "Map Options". Tableau will do it for you.

Mandy

Anonymous user

Hi Mandy -
Can you give more guidance on how to access those boundaries and load them such that they can be used to create a thematic map?
SCR

James Baker

Posts: 2175

Joined: December, 2007

Zip code boundaries are a layer that can be turned on in the Map Options card (View -> Other Cards -> Map Options if it's been turned off).

While you can map polygon data, you cannot create a thematic map based upon the map layers provided.

Btibert3

Posts: 32

Joined: November, 2008

Hi guys,

I was just about to ask this question, and realized there is a good dicussion going on.

I am a complete newbie to Tableau, but have mapped data in Arcview. What do I have to fill in polygons here in Tableau - right now, I just want to fil in states.

I know you gave a great example, I just need to be explained the process. Essentially, what data files do I need to bring over from Arcview and how do I attach the data in Tableau?

Many thanks for your insight,

Brock

Anonymous user

All,
Great posts. It appears I am missing the step of how to convert the census file format (Arc/info Arcview) into something that I can inport to Tableau (access, xls, etc...) eg. election.xls
Thanks for your help on this.
-Chris

Cush

Posts: 2

Joined: February, 2009

As an update: I was able to hack through the ascii file to pull the information for Georgia and create zip code level detail. Still looking for a quicker way to convert those census files into Access. It was quite tedious and would not be time effective for all 50 states.

AttachmentSize
Georgia Zip Level.twbx15.45 MB

Dan Gerena

Posts: 51

Joined: November, 2007

It'd be nice if Tableau could format the files and make them centrally downloadable to each of us, so we don't all independently recreate the wheel. Tracking them down and manipulating is the hard part of this process.

marjenmiller

Posts: 30

Joined: February, 2009

Chris (Cush) - can you explain how you crunched the numbers from ASCII and how you got it into Tableau?

joemako

Posts: 882

Joined: September, 2008

I wrote a macro that rearranges the raw data in the 104 ASCII files, merges them together and puts that data into a normalized Access database. Sometime in the next 12 hours I'll upload the .mdb file.

joemako

Posts: 882

Joined: September, 2008

An Access 2000 .mdb file of the zip code data is being uploaded right now (50 MB) to my server. I'll have the link here in about an hour. It has 52 States with 50,080 Polygons with 5,732,964 Points. The compressed file expands to about 400 MB.

Please be aware that this data is at least 9 years old, and that is is not recommended to have Tableau draw all the zip code polygons for all the states at once.

Sometime this week, I plan on making a short video tutorial on how to use this data.

If you have any isuess or comments you can contact me here or via http://twitter.com/joemako (or if you need help reformatting your data for use in Tableau)

joemako

Posts: 882

Joined: September, 2008

The zip code data can be downloaded from http://joemako.com/files/ZipData.rar

This is a .rar file not .zip, you will need a application like WinRAR 3.80 that can be downloaded from http://www.rarlab.com/download.htm

Btibert3

Posts: 32

Joined: November, 2008

Hi Joe,

I know am asking alot considering you already uploaded a database for us to use, but have you had a chance to create a brief tutorial for newbies like me on how to associate data with the polygons?

Thanks again!

Brock

marjenmiller

Posts: 30

Joined: February, 2009

Good idea Brock. I could use it too.

marjenmiller

Posts: 30

Joined: February, 2009

Good idea Brock. I could use a mini write up too on how to use the data. Thanks!

joemako

Posts: 882

Joined: September, 2008

I spent some time this weekend working with the data, trying to find a meaningful way to use the zip code data from 2000, and was unable to.

I did some more searching and found this thread, http://www.reddit.com/r/programming/comments/7z8al/where_can_one_get_zip... where people agreed that the 2000 data is not useful. A good read if you are thinking about using zip code boundaries.

The thread did point to http://www.census.gov/geo/www/tiger/ that has data that looks to be both more current and more useful.

I will take a look at using SharpMap, http://www.codeplex.com/SharpMap to extract the data from the ESRI Shapefiles to make it usable in Tableau.

I really wanted zip boundaries to work, but the more I learned about them, the more I realized that the way Tableau maps zips is more effective on every front than anything I could put together with zip boundary data.

I am going to take a look at compiling state boundary data, dealing with islands in states, and work on putting together an easy process for creating custom region boundary data.

Sorry no video yet :( Marc Rueter did a really good job, and I am still learning off of his work. When I can create a map like that from scratch, then I will have video.

Thank you for your patience.

Anonymous user

Can anybody point me in the direction of some more detailed training on how to establish a set of boundary overlays on a map? I am looking to establish some mapping visualizations that depict school district and school boundaries and where students live in relation to the schools' boundaries.
Thanks,
Danny Combs

joemako

Posts: 882

Joined: September, 2008

Danny Combs,

I would be glad to help out, I have some free time this upcoming week. Feel free to contact me via email joemako [at] gmail [dot] com or on twitter http://twitter.com/joemako.

sbenners

Posts: 6

Joined: April, 2009

Wow, there is obviously a lot of buzz going on in this area. I'm new to using Tableau and I'm currently doing an R&D engagement with our data. I looked at the elections.xls file in the packaged workbook example provided towards the top of this thread. It has two fields in it that I'm interested in: PolygonId and PointOfOrder. These two fields are obviously critical to creating the Polygon Map example which is very similar to what I'm looking to create. I don't have this data and I'm interested in obtaining it. Currently I've got city, state, zip code, and lat/long coordinates...how do I go about obtaining these last two fields though? Anybody know?

joemako

Posts: 882

Joined: September, 2008

sbenners,

PolygonId is a unique identifier number for a polygon, most states have only one, but some states may have more, because of islands.

PointOfOrder is a unique identifier number for each point/vertex of a polygon sorted in order to drawn, more complex states will have more. Each PointOfOrder will have a Lat/Long coordinate.

If you would like to discuss more details/help on creating these visualizations yourself, contact me on twitter or via email (see above).

sbenners

Posts: 6

Joined: April, 2009

Within the elections data there are 5 states ('NY','MN','VA','MI','TX') that have more than one Polygon Id associated with them. In my smaller subset of data I'm trying to determine which Id to use with each Latitude and Longitude coordinate since my set of coordinates are different from the ones provided in the elections data. It feels like it is a guessing game (i.e. numbers just being close) rather than a science. I wish the technical support at Tableau would provide more insight and direction in this area since the Polygon File Structures require both the coordinates as well as the Polygon Id and Point of Order.

sbenners

Posts: 6

Joined: April, 2009

Well, I went through the steps to populate both the PolygonId and PointOfOrder fields. It looked good when I finished it. However, when I applied my source data to a map, it looked like crap. Polygon structures were being painted all over the place and it looked like the work of a little kid in art class. I sure wish someone from Tableau could provide further insight on the process of populating these fields, their validation, and the science behind applying them. I know it's something that I'm doing wrong, and I'm sure I could get things cleared with a little assistance from the creators of the software.

joemako

Posts: 882

Joined: September, 2008

OK, let’s see if I can approach this in another way. Instead of looking at states, let’s take a look at some simple shapes, and relate it to working with states.

Notes on the attached image "layout.png":
- Shapes A and B are single contiguous polygons (Similar to a state like Utah)
- Shape C has two polygons (Similar to a state with an island, like Texas)
- Points are ordered like a connect the dots game, if you change the order, the shape will be different.
- For each polygon, the last point is a repeat of the first point to close the loop (this may not be required, but I think it is a good practice)
- The values on the X and Y line can be used similar to Lat/Long

Notes on the attached Excel file "shapes.xls":
- On the "Shapes" tab, you can see 3 different Shapes (A,B.C), 4 PolygonIDs (1,2,3,4), 19 PointOrders (1-19), and X and Y coordinates for each point. All this data came from the layout.png image. (similar to state polygon/point data in that states are like shapes)
- On the "Data" tab, this would be your source data, Shape A has 1 store located in it, Shape B has 2 stores, and Shape C has 3 stores. I've also included sales figures for each store.
- The "Combined" tab is what you get after merging these two data sets together, (I'll get into how it was made next).

On the "Shapes" tab, 5 rows describe Shape A, 6 describe B, and 8 describe C.

Taking a look at the "Combined" tab, you see 41 rows of data, 5 for Shape A, 12 for B, and 24 for C.

So 5 rows became 5, 6 rows became 12, and 8 rows became 24. The math is (5*1)+(6*2)+(8*3) = 5+12+24 = 41 The 1,2,3 came from the number of stores in the corresponding shape.

Attached is an image "query.png" that shows the two tabs and the query that generated the "Combined" tab.

The query is simply a query with both tables, joined on Shape that include the * from Shapes and the fields you want from your data.

When we have Tableau connect to the Combined data source, we are able to see the result, attached as "Shapes.twbx".

I made a couple of tabs that provide different views of the three tabs.

If any of this does not make sense, please let me know what part(s) I was not clear on.

I have also attached a "States.xls" that came from the elections.xls file that has the same structure as the "Shapes" tab (State abbr. instead of Shape, and Lat/Long instead of X/Y)

Best Regards,
Joe Mako

AttachmentSize
States.xls675 KB
Shapes.xls18 KB
shapes.twbx82.17 KB
query.PNG93.82 KB
layout.png15.44 KB

joemako

Posts: 882

Joined: September, 2008

I just want to make sure my stance on this type of data visualization, using state polygons, is clear.

I do not think this is the best way to do this.

You can see some examples on http://flowingdata.com/2009/05/04/10-visualizations-for-number-of-days-t... where bar graphs with Tableau are easier to gain understanding of the data than a state polygon map.

Sometimes it does work out well at the county level, see http://www.nytimes.com/interactive/2009/03/10/us/20090310-immigration-ex... as most counties are a similar size in area.

Stephen Few clearly states best practices when geocoding data at http://www.perceptualedge.com/articles/visual_business_intelligence/geog... that I strongly agree with.

As always, just because you can do something, does not mean you should, and if it is worth doing, it is worth doing right.

Other than that, I do see a future use for polygons in Tableau, but not with state shapes. I have not fully formed or tried my ideas out yet, but when I do, and if they are useful, I will start another thread.

Anonymous user

I'm a newbie to Tableau too. I'm looking to synchronize Tableau with ESRI products and was wondering if it was possible to add in my own polygons that I create, not just the predefined States, zip codes, etc. from Tableau. Do I have to connect to an server with ESRI supported data?

Anonymous user

I am trying to build a densities map, similar to what I would normally do in ArcGIS. I am having some difficulty since I am trying to do it at the zip3 or SCF level. Any idea where I can find a polygon for this?

haniar

Posts: 12

Joined: July, 2009

Hi All,

Tableau geo data is very much geared towards US. Does anyone know where I can get country polygones, or even internation cities such as London lat/lon coordinates.

Julius

Posts: 85

Joined: May, 2009

Hello all, just wanted to see how Tableau is doing with any new capabilities related to choropleth or thematic mapping. I've seen the point density maps as well as examples of shape drawings but was looking to see if there are more intuitive ways to create something similar to the attached image (county_choropleth.gif)

Also would like to plug a couple of blogs related to mashup efforts around GeoBI:
http://blog.thematicmapping.org/
http://highearthorbit.com/geocommons-cloudmade-integration/

Thanks.

[UPDATE]: Sorry for being redundant, Joe. I just realized that you had a similar posting on 5/4/09 regarding the use of choropleth maps in NYTimes. That's exactly what I was inquiring about and wanted to see when we can expect this feature. The bubble map is a great start but with dense amount of data bubbles start to overlap and lose detail - would love the thematic option. Thanks!

AttachmentSize
Thematic_Mapping_Engine.pdf7.07 MB
county_choropleth.gif83.25 KB

Anonymous user

Stephen Few explictly endorses thematic maps/choropleth maps!

"Two approaches to displaying quantitative information on maps usually work best: variations in color intensity, in size, or both. The map below, which I borrowed from Gretchen Peterson’s excellent new book GIS Cartography: A Guide to Effective Map Design, illustrates the use of color intensity for displaying quantities. Imagine that this is a country divided into provinces, and that various intensities of the color orange are being used to encode average household income—the darker the color the greater the income. This approach displays an aggregate measure for each province rather than a measure for each household. Geographical displays of this type are called choropleth maps."

Why can Tableau not:

1. Let us apply our own "data layers" to maps as you do with population, et al.?
2. View county borders at a national view?

I sense that what you are saying is that this would be really hard to accomplish in Tableau's current architecture. That is not a reason not to do it. I'd be happy with choropleth maps separate and apart from Tableau's main mapping construct. No need to worry about drawing marks on top of thematic maps. Just let me have the option of thematic maps at the state, county, and MSA level for the U.S. and keep it simple for me and yourselves. Give us what we want and what our users expect, in other words.

Anonymous user

Any update on this. This feature would be really useful to roll out across all 50 states.

Anonymous user

Does anyone know a way to have the Polygon Id (or state name) show on the map so people can tell the names of the states.

Anonymous user

hello!

hey people please do not sleep and let us know how polygons of shape files regarding non us countries can be
included in your programm. you have such fantastic software product which also we in europe would like to use.

please prevent me to make a visit in your office with the rest of my crew - see photograh below (irian jaya 1998)

AttachmentSize
sa.jpg176.14 KB

Chris Cammack

Posts: 15

Joined: December, 2009

Has Tableau decided to address this yet? It is frustrating to have such a powerful program be limited by a seemingly fixable problem? The anon user from the 25th is right on the money.

Douglas

Posts: 18

Joined: January, 2010

Does anyone know of a source for US county polygons? Or of a way to convert this KML into Tableau-consumable data? http://www.gelib.com/us-counties.htm

Thanks,

Douglas

(add my vote to the request for full chloropleth support in Tableau!)

Anonymous user

was there any response to the posting by Douglas? i would like to get the same data - US county polygons. Please advise if there has been any updates on this.

Douglas

Posts: 18

Joined: January, 2010

Here's the data in SVG format. http://upload.wikimedia.org/wikipedia/commons/5/5f/USA_Counties_with_FIP...

Would be lovely if someone would convert one of these to Tableau-happy format.

Anonymous user

Douglas,

That is a great SVG file, but none of the coordinates are lat/long, they are pixel x/y. If you do not want to use lat long, then it might be useful. Is that what you are asking? Even if the boundry shapes are extracted into a CSV, you will still need to process the data (see joemako's comment above from April 30th, 2009).

You may be looking for the Shapefiles from http://www2.census.gov/cgi-bin/shapefiles2009/national-files

Anyone have a resource for converting Shapefiles to CSV?

Be aware that polygon files are not very powerful in Tableau, nor are they quickly rendered in large scale. You may be better off setting up a WMS server, and use data layers.

Another better choice is use a bunch of marks like in the video at http://www.tableausoftware.com/videos/advanced-mapping-techniques starting around 33:00, but that approach is only available against a data source like SQL Server because the RAWSQL function uses geom.

Anonymous user

OK, as kind of a newbie here I have pieced together what needs to be done to use polygons for thematic maps based roughly on zip codes (understanding that zip code polygons roughly approximate delivery areas).

I downloaded the latest zip code shape files from here

http://www.census.gov/geo/www/tiger/tgrshp2009/documentation.html

and I used SAS to extract the raw data fdrom the shape files.

However, I don't see anything that seems to corresponds to the "order" or "point order" field that indicates the order in which the polygon line segments are connected.

Everything else seems to be there.

Anyone know how I should decipher or interpret the data from the shape file?

Variables extracted, with sample data, are

X, Y, SEGMENT, ALAND, AWATER, CLASSFP, FUNCSTAT, INTPTLAT, INTPTLON, MTFCC, ZCTE5CE

-112.617833, 44.241085, 1, 1667073388, 272487, B5, S, +44.2171357, -112.4327143, G6350, 83423

How do I get and "ORDER" out of this?

Thanks.

stud_student

Posts: 5

Joined: March, 2010

BUMP.

I'm trying to do the same thing - figure out / extract the point orders from the census data.

joemako,

Excellent post explaining how point orders work.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options

Syndicate content Subscribe to the comments on "Polygon Files Structures - Solid Color Maps"