Evaluating the Feasibility of ChatGPT for Mapping Building Attributes

In the past we have written about using Multimodal Large Language Models (MLLMs)  like  ChatGPT for coding of models and also  analyzing images. One advantage we see for MLLMs is that unlike traditional approaches that require extensive expertise in computational analysis, such as computer vision and deep learning, MLLMs leverage pre-trained capabilities that simplify the analytical process. This accessibility enables a larger group of researchers to incorporate MLLMs in their analyses when it comes to studying the form and function of cities at scale. To this end, we (Qingqing Chen, Linda See and myself) have new book chapter entitled "Evaluating the Feasibility of ChatGPT for Mapping Building Attributes" published in the open access book: "Geography According to Foundation Models" edited by  Krzysztof Janowicz, Rui Zhu, GengchenMai, Song Gao. Yingjie Hu, Zhangyu Wang, Ling Cai and Lauren Bennett.

In this chapter we evaluate the potential of MLLMs, in our case ChatGPT, to extract building attributes (e.g., age, use and height) from Mapillary street view images.  We find that ChatGPT was good at extracting some information and less good in other cases. For example it identified correctly 87% of the residential buildings. We also discuss ways to improve the results (e.g., using higher quality street view images, altering and refining the prompts). If you wish to find out more about our findings we encourage you to read the chapter. To give you a better sense of this research, below we provide the abstract to the paper, our case study area along with our workflow and a sample of the results. Finally at the bottom of the post, you can find the full reference to the chapter along with a link to it. 

 

Abstract:

With increasing rates of urbanization, many challenges are emerging regarding urban sustainability such as the energy usage of buildings. Coinciding with this is the growing attention of urban climate models for energy demand estimation and climate adaptation strategies. However, the applicability of these models is constrained by the lack of detailed urban surface information. Therefore, creating comprehensive datasets that capture urban surface information at a granular scale is crucial for responding to our rapidly urbanizing world. Recent advancements in Multimodal Large Language Model (MLLMs) have opened new opportunities in urban studies, offering accessible methods for information extraction. In this chapter we explore the feasibility of ChatGPT to extract building attributes from images. Taking New York City as a case study, we collect building images from Street View Imagery and process them through ChatGPT by posing specific questions to extract building attributes (e.g., height, functions, age). These attributes are then compared with authoritative data. The proposed method helps address the current dearth of fine-grained surface data on urban issues, therefore enhancing the accuracy and utility of urban climate models. Overall, this study demonstrates the practical applications of ChatGPT in geographic knowledge extraction, advancing the understanding of MLLMs in geographic contexts, and more broadly to the discourse on Artificial Intelligence (AI) in urban modeling and climate science.

The spatial distribution of Mapillary images within the study area, shown on the left, and the distribution of images by variance showing increasing image quality on the right.

An overview of the research workflow.

Comparison of the building period of construction from the ground truth data and the classifications from ChatGPT. (a) A confusion matrix which details the distribution of buildings classified within each period by ChatGPT compared to the ground truth data; (b) A chord diagram illustrating the patterns of agreement and confusion among the categories.

Comparison of building type classifications. (a) A confusion matrix detailing the distribution of ChatGPT’s classifications against the hand labels from experts; (b) A chord diagram illustrating the proportion of classifications for each building types as labeled by experts compared to ChatGPT’s classifications.

A comparison of building heights from ChatGPT and the NYC Open Data. (a) The correlation of height between the ground truth and ChatGPT; (b) The distribution of ground truth heights and the predicted heights; (c) The difference in the heights.

Full Reference:

Chen, Q., See, L. and Crooks, A.T. (2026), Evaluating the Feasibility of ChatGPT for Mapping Building Attributes,  in Janowicz, K., Zhu, R., Mai, G., Gao, S., Hu, Y., Wang, Z., Cai, L., and Bennett, L. (eds), Geography According to Foundation Models, IOS Press, Amsterdam, The Netherlands, pp. 107-120. (pdf)

New Paper: Exploring Fear in Urban Environments

In the past we have written about how we have used social media to study a plethora of topics with respect the the form and function of cities among many other things. But one thing we have not explored is fear and more specifically fear of crime and how this can be mined through geosocial media. 

This has now changed with a new paper entitled "Exploring Fear in Urban Environments: Place and Space Analysis of Social Media Data" which has recently been published in Applied Geography.  In this paper, Ying Zhou and myself extract fear related posts from social media and examine the places and spaces where people experience fear, as well as the factors that contribute to it in New York City. 

We do this by utilizing Natural Language Processing (NLP) techniques for sentiment and text analysis, including a RoBERTa-based emotion classification model and the BERTopic model for topic modeling. The former model narrowed the raw data to those with the dominant emotion of fear, and the latter analyzed space- and place-related features that contribute to the fear sentiment. Then, the selected social media data were analyzed using spatial clustering methods (i.e., Hotspot Analysis (Getis-Ord Gi*) and Local Moran’s I) and compared with urban crime data for weekly trends and spatial patterns. As such the paper has the following research objectives:

1. exploring places where people expressed fear through social media; 
2. making comparisons between safety-related fear and crime from the perspective of both time and space; 
3. extracting urban environmental and social features that lead to fear.

If this sounds of interest, and you wish to find out more with respect to our findings, below you can read the abstract to the paper, see some of the figures which describe our research methodology and results while at the bottom of the post you can find a link to the paper itself. Finally the code we utilized in the paper can be found at https://osf.io/y7xfc/overview.

Abstract:

One goal of creating livable cities is to enhance public safety. While previous research in urban studies has focused on correlations between physical environments and crime, it has typically relied on criminal statistics. However, fear of crime is an emotional response to perceived risks rather than a direct reflection of crime levels, so it cannot be analyzed solely by crime data. Additionally, urban planning today has gradually shifted its focus from a top-down to a bottom-up approach, making it essential to understand and foster spaces where residents feel safe. This research examines the spaces and places where people experience fear, as well as the factors that contribute to it, in New York City. We utilized social media data to gather people’s expressions of the city and identified posts expressing fear emotion using the RoBERTa-based model and a rule-based classifier. Then, the selected social media data and crime were compared temporally by weekly trends and spatially by clustering methods (i.e., Hotspot Analysis (Getis-Ord Gi*) and Local Moran’s I). The results show that their temporal and spatial patterns partially have limited alignment. To delve into the origins of fear, we extend our analysis by adopting BERTopic to identify topics and summarize them into themes (e.g., places, transportation, people, others) to understand the bottom-up emergence of fear, thereby informing a people-centered approach to research on urban issues. 

Keywords: Social media; Natural language processing; Sentiment analysis; Urban environment.

Methodology framework.

An example of textual analysis on fear-related tweets: from machine-generated topics to human-interpreted themes describing fear in NYC.

Weekly trends comparison between safety-related fear and violent crime.

Clustering features analysis by the method of hotspot analysis (Getis-Ord Gi∗).

Full Reference: 

Zhou, Y. and Crooks, A.T. (2026), Exploring Fear in Urban Environments: Place and Space Analysis of Social Media Data, Applied Geography, 192: 104051 (pdf)

New Paper: Connecting senses: The cross-modal associations between smell and vision in understanding urban environments

In a previous post we wrote about how one can mine social media to uncover smells and how they shapes peoples perceptions of urban spaces. Building off this work we (Qingqing Chen, Ate Poorthuis and myself) have a new paper entitled "Connecting senses: The cross-modal associations between smell and vision in understanding urban environments" published in Geographical Analysis. 

In this paper we build upon this but at the same time move away from social media to explore the relationship between smell and vision. We do this utilizing street view imagery from New York City, in this case we are using Mapillary. A subset of these images were labeled with pre-defined smell categories (e.g.,  ‘Nature’, ‘Food’, ‘Transportation & Fuel’) in order to develop a deep learning smell classifier capable of classifying perceived smells from the images. We then use  advanced image processing techniques (e.g., ResNet50, VGG16, Inception-V3, MobileNet and EfficientNet.) to extract visual cues from the street view imagery to predict smells. 

If this sounds of interest, below we provide an abstract to the paper along with some of the figures which show our workflow and accompanying results. At the bottom of the post you can see the full referece to the paper, while at https://figshare.com/s/94cdec3b14c206e6d225 we provide our code to allow others to replicate or extend this to other areas. 

Abstract:   

Smell is a crucial yet understudied sensory dimension in urban environments, bridging tangible elements (e.g., exhaust, flowers) with intangible impacts on emotions, social interactions and well-being. While geographical and urban research increasingly acknowledges multisensory experiences, much of geospatial analysis still emphasized the visual dimension. This research advances spatial thinking by examining cross-modal associations between smell and vision in urban environments. Specifically, we utilize advanced image processing techniques to extract visual cues from street view imagery (i.e., Mapillary) and apply causal analysis to examine their effects on smell expectations recorded from participants. The results show that visual cues can predict smells in straightforward urban settings (e.g., parks or less densely populated areas). However, in complex urban environments, the predictive power of visual cues diminishes as diverse and overlapping scents obscure specific smells, even in visually distinct areas. These findings underscore the importance of a multisensory approach in urban analytics, enhancing our understanding of the interplay between sensory experiences and informing urban design strategies that integrate multiple senses to create engaging and inclusive environments. This is especially important for individuals with sensory impairments, such as anosmia or visual impairments, who rely on other senses to compensate for their perception of urban environments. 

Keywords: Smell and Vision; Cross-modal Associations; Multisensory Experiences; Image Processing; Street View Imagery (SVI)

An overview of research workflow.

The visual feature extraction framework based on key patterns identified from questionnaires.

Spatial distribution of images with different perceived smells and example participant notes identifying potential smell sources responsible for perceiving the dominant smells.

Identified important features for smell categories that are relatively indirect to be inferred from visual cues. Left: Identified important features; Right: Examples of feature  contribution in individual images.

Full Referece: 

Chen, Q. Poorthuis, A. and Crooks A.T. (2026), Connecting senses: The cross-modal associations between smell and vision in understanding urban environments, Geographical Analysis, 58 (3): e70046.. Available at https://doi.org/10.1111/gean.70046. (pdf)

Research Updates: AAG 2026

At the AAG Annual meeting this year, two of my students gave talks about their ongoing research. Ying Zhou presented her work with a talk entitled "Exploring the Relationship between Urban Morphology and People’s Emotions." In this talk, Ying showed how one could mine social media posts to gain a sense of how different emotions are spatially spread around a city using New York city as a case study. If this sounds of interest, below you can see the abstract of the talk, the research methodology and a sample of the results.  

Abstract: 

Urban morphology records physical information about spatial patterns (e.g., streets and land use) and their evolution over time, as well as human settlement information. People who live in or visit a city gain experiences through interaction with its spatial patterns, and these experiences influence people’s emotions. Therefore, it is necessary to explore the spatial relationships between urban morphology and people’s emotions. Taking New York City as a case study, this research uses social media data to obtain and locate people's emotions in different parts of the city. To extract the emotion relating to specific space, we use the RoBERTa-based model to label texts in social media with six primary emotions (i.e., happiness, sadness, fear, anger, surprise, and disgust). We then used DBSCAN to identify spatial clustering features of these emotions. Finally, we compared the clustered emotions with urban morphology (both in terms of both its form and function) and how such emotions evolve and change over a span of five years. Such analysis reveals the relationship between people’s emotions and broader setting that they inhabit (i.e., the city). Moreover, these works offer bottom-up insights into how urban morphology shapes people’s feelings, which can serve as feedback for urban planning and management.

 

Keywords: Urban Morphology, Emotion Detection, Spatial Analysis, Urban Studies.

While in another talk, Boyu Wang continues to add new functionality to the Mesa, a python agent-based modeling toolkit, this time in the form of utilizing large language models for agent-based decision making, with a talk entitled "Mesa-LLM: Generative agent-based modeling with large language models empowered agents" 

If this sounds of interest, below you can see the abstract of the talk, along with the Mesa-LLM architecture. While further details about Mesa-LLM can be found on Boyu's GitHub page: https://github.com/mesa/mesa-llm.

Abstract 

Agent‐based models (ABMs) have long been used to examine how individual behaviors give rise to aggregated social and spatial phenomena. Mesa, an open source ABM library in Python, provides modular components and browser based visualization to create and analyze agent based models in the PyData ecosystem. Agents’ behaviours in these models are often governed by rule-based decisions. The recent advancements of large language models (LLMs) have created a new paradigm, namely generative agent-based modeling, where LLMs are integrated as decision-making engines so that agents can communicate, negotiate, and decide based on natural language. In this paper, we introduce Mesa-LLM, an LLM extension to the Mesa framework. Its modular design allows users to customize reasoning, memory and planning components and plug in different LLMs (e.g., GPT, Gemini, Llama). We demonstrate Mesa-LLM through Epstein’s civil violence model. In contrast to the classical model where agents act based on calculated probabilities and pre-defined thresholds, agents through Mesa-LLM have their decisions articulated in natural language. This demonstration shows how an archetypal ABM can be enriched by language-based decision making to explore complex social dynamics such as protest escalation. Through this simple example, we highlight how incorporating LLMs into ABMs opens new possibilities for geographers to model human behavior from the bottom up by leveraging generative artificial intelligence (GenAI).

 Keywords: Agent-Based Modeling, Large Language Model, AI Agent, Python.

References 

Wang, B., Frisch, C., Nair, S., Kazil, J. and Crooks, A.T. (2026), Mesa-LLM: Generative Agent-Based Modeling with Large Language Models Empowered Agents, The Association of American Geographers (AAG) Annual Meeting, 17th –21th March, San Francisco, CA. (pdf)

Zhou, Y. and Crooks, A.T. (2026), Exploring the Relationship between Urban Morphology and People’s Emotions, The Association of American Geographers (AAG) Annual Meeting, 17th –21th March, San Francisco, CA. (pdf)

PySGN: A Python package for constructing synthetic geospatial networks

In previous posts, we have written about the generation of synthetic populations based on real world locations, and how such populations can have various types of networks associated with them. We have also written about network generation techniques in the past and keeping with this line of research, Boyu Wang, Taylor Anderson, Andreas Züfle and myself have a new paper in the Journal of Open Source Software entitled "PySGN: A Python Package for Constructing Synthetic Geospatial Networks"

In this paper we introduce a Python package that can generate geospatial networks which we have called PySGN (Python for Synthetic Geospatial Networks). For readers not familiar with geospatial networks, to quote from the online documentation we have put together:  

Geospatial networks are a type of network where nodes are associated with specific geographic locations. These networks are used to model and analyze spatial relationships and interactions, such as transportation systems, communication networks, and social interactions within geographic constraints. By incorporating spatial data, geospatial networks provide insights into how location influences connectivity and network dynamics.

PySGN generates synthetic geosocial networks that mimic the spatial relationships observed in real‑world networks as it embeds nodes in geographic coordinate space, modifies connection rules to decay with distance, and allows users to incorporate clustering and preferential attachment while respecting spatial constraints. Online we provide examples of creating Geospatial Erdős-Rényi, Watts-Strogatz & Barabási-Albert Networks along with ways to sample points based on a specified bounding box or specific polygon boundaries (examples of which are shown below). 

The package is intended for researchers and practitioners in fields such as urban planning, epidemiology, infrastructure resilience and social science who require robust tools for simulating and analyzing complex geospatial networks. In addition to the paper, we have also made available extensive documentation (along with examples of the various network types) at https://pysgn.readthedocs.io/en/ 

Examples of Geospatial Erdős-Rényi and Watts-Strogatz Networks.

Example of Geospatial Barabási-Albert Network based on different ordering strategies for how nodes are added to the network.

Examples of sampling points based on a bounding box or a specific set of polygons

Full Referece: 

Wang, B., Crooks, A.T., Anderson, T., and Züfle, A. (2026), PySGN: A Python Package for Constructing Synthetic Geospatial Networks. Journal of Open Source Software, 11(119), 9346, https://doi.org/10.21105/joss.09346 (pdf)