MAKING SENSE OF PLACES FROM MILLIONS OF TWEETS
SensePlace2 / SensePlace3 was a project we collaborated on with research groups at The Pennsylvania State University in the Geography Department, College of Information Sciences and Technology, and Computer Science and Engineering Department. This project was funded by the US Department of Homeland Security (DHS) and the US Army Engineer Research and Development Center (ERDC), Geospatial Research Laboratory. Our goal was to analyze place-time-attribute information found in Twitter tweets and make sense of places from millions of tweets. Not only did we build a system to map and analyze the small percentage of tweets where the user enabled their location, but we also led the way to analyze the places mentioned in the tweets. Our work was one of the first to describe using tweets and their location data for situational awareness during emergencies. It also led to important discussions in the research and emergency response communities of the ethics of location-based data in crisis situations. The videos below highlight the system, the problem it addresses, and its goals.
Video 1: SensePlace2: Visual Analytics and Big Data for Spatiotemporal Sensemaking
Video 2: SensePlace2 Functionality – October 2012
Video 3: SensePlace2 – Geo Twitter Analytics
ACCOLADES
Video 4: IEEE VAST 2021 Test of Time Award acceptance speech.
- Pezanowski, S., MacEachren, A. M., Savelyev, A., & Robinson, A. C. (2017). SensePlace3: a geovisual framework to analyze place–time–attribute information in social media. Cartography and Geographic Information Science, 00(00), 1–18. https://doi.org/10.1080/15230406.2017.1370391
- MacEachren, A. M., Savelyev, A., Pezanowski, S., Robinson, A. C., & Mitra, P. (2014). Report on New Methods for Representing and Interacting with Qualitative Geographic Information, Stage 2: Task Group 3: Social-focused Use Case. http://www.dtic.mil/get-tr-doc/pdf?AD=ADA618993
- Savelyev, A., MacEachren, A. M., Pezanowski, S., Karimzadeh, M., Luo, W., Nelson, J., & Robinson, A. C. (2014). Report on New Methods for Representing and Interacting with Qualitative Geographic Information, Stage 2: Task Group 4 Message-Focused Use Case. http://www.dtic.mil/get-tr-doc/pdf?AD=ADA622198
- Savelyev, A., Pezanowski, S., Robinson, A. C., & MacEachren, A. M. (2012). Report on Component 2-Designing New Methods for Visualizing Text in Spatial Contexts. http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA580753
- Robinson, A. C., Savelyev, A., Pezanowski, S., & MacEachren, A. M. (2013). Understanding the utility of geospatial information in social media. ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management, 918–922. http://www.scopus.com/inward/record.url?eid=2-s2.0-84905656005&partnerID=tZOtx3y1
- MacEachren, A. M., Jaiswal, A., Robinson, A. C., Pezanowski, S., Savelyev, A., Mitra, P., Zhang, X., & Blanford, J. (2011). SensePlace2: GeoTwitter analytics support for situational awareness. VAST 2011 – IEEE Conference on Visual Analytics Science and Technology 2011, Proceedings, 181–190. https://doi.org/10.1109/VAST.2011.6102456
- MacEachren, A. M. . M. a. M., Robinson, A. C. ., Jaiswal, A., Pezanowski, S., Savelyev, A., Blanford, J., & Mitra, P. (2011). Geo-Twitter Analytics: Applications in Crisis Management. In A. Ruas (Ed.), Proceedings of the 25th International Cartographic Conference (pp. 3–8). https://icaci.org/files/documents/ICC_proceedings/ICC2011/Oral%20Presentations%20PDF/C3-Geovisualisation%20and%20data%20exploration/CO-244.pdf
BRIGHTWORLD LABS CONTRIBUTION
We were fortunate to collaborate from the beginning of this project. We:
- Met with key DHS and US Army Corps of Engineers to understand their problem.
- Contributed to novel research ideas to solve the challenge of making sense from big social media data.
- Implemented technical solutions involving:
- Natural language processing to extract place, people, and organizations text.
- Information Retrieval and search engine technology to return most relevant tweets to the user searches.
- Geographic Information Retrieval to parse and geocode place mentions
- Full stack geovisual web application development involving Postgresql/PostGIS, Java web services, and many JavaScript mapping, visualization, and Web 2.0 client APIs to create a human-in-the-loop geovisual analytics system.
- Devised many strategies to handle, process, search, and display nearly 30 million tweets per month in near real-time.
- Presented at international academic conferences and many times to top project officials at the funding agencies.
- Wrote academic articles for top international publication outlets.
The following GeoVISTA Center article describes our work with the Institute for Computational and Data Sciences (ICDS) to overcome the technical challenges of dealing with this messy big data source.
CONCLUSION
Posed with the problem of making sense from place-time-attribute information in big social media data, we helped create a geovisual analytics system to allow a human-in-the-loop sense-making process. This project also spun off GeoTxt, a web API for geoparsing and geocoding of place names in text documents, that we contributed heavily towards.