The first paper was with Qingqing Chen and Linda See and was entitled "Using New Sources of Data for Urban Climate Modeling Generated through MLLMs on Street View Imagery. "As the title might suggest, this paper was about how one can leverage multi-modal large language models (MLLMs) to extract information on building height, age and function from street level photographs. We demonstrate this using street view images from Mapillary and than ask ChatGPT to estimate the building height, age and function and compare the results to authoritative data sources. If this sounds of interest, below you can see the abstract to the paper, some if the figures (i.e., the work flow and prompts) while the results can be seen in the attached paper (see the link below).
Abstract:
Urban climate and energy balance models require data on the form and function of buildings, but high resolution spatially explicit data sets are often lacking. Here we demonstrate how multi-modal large language models (MLLMs) can be used to extract information on building height, age and function from street level photographs for New York City. A workflow is presented that illustrates the approach, with initial results indicating that the building function can be identified with good accuracy while moderate accuracies were obtained for building heights and age. Suggestions for how to improve these accuracies are also provided.
KEYWORDS: Buildings, ChatGPT, Multi-modal Large Language Models (MLLMs), Mapillary, Street View Images (SVI).
An overview of research workflow.
The detailed description of multi-step prompting and an example of extracted building attributes information.
Full Reference:
Chen, Q., See, L. and Crooks, A.T. (2025), Using New Sources of Data for Urban Climate Modeling Generated through MLLMs on Street View Imagery. In Cramer-Greenbaum, S., Dennett, A., and Zhong, C (eds.), Proceedings of the 19th International Conference on Computational Urban Planning and Urban Management (CUPUM), London, UK. (pdf)
Modeling human behavior continues to be a significant challenge for the field of agent-based modeling, and one that prohibits the development of comprehensive empirical ABMs for urban applications, such as Urban Digital Twins. However, two recent methodological advances offer the potential to transform empirical agent-based models.
Early evidence suggests that large-language models (LLMs) can be used to represent a wide range of human behaviors, with models responding in realistic ways to given prompts. Indeed there is already a flurry of activity that focusses on implementing LLM-backed agents -- i.e. agents who are controlled by LLMs. At the same time, the concept of the foundation model is also being applied in domains beyond text analysis. Of particular interest are geospatial foundation models that automatically encode spatial data in such a way as to associate different spatial objects in numerous and nuanced ways that have otherwise alluded manual classification schemes. Taken together, these two technologies offer considerable potential for a new generation of agent-based models that contain agents who can behave in response to spatial and social prompts in a way that is realistic and has so far proven impossible to replicate using manually-programmed behavioral rules.
This paper presents a discussion of the state of the art in both LLMs and geospatial foundation models in the context of their potential role in agent-based modelling. It discusses the transformational potential of these technologies and outlines the critical questions that need to be addressed before they can be used to create robust, reliable and trustworthy models for empirical policy applications that support decision-making.
KEYWORDS: Agent-based Modeling; Large language model; Geospatial foundation model; Urban Modeling.
Full Reference:
Malleson, N., Crooks, A.T., Heppenstall, A. and Manley, E. (2025), Enhancing Spatial Reasoning and Behavior in Urban ABMs with Large-Language Models and Geospatial Foundation Models. In Cramer-Greenbaum, S., Dennett, A., and Zhong, C (eds.), Proceedings of the 19th International Conference on Computational Urban Planning and Urban Management (CUPUM), London, UK. (pdf)
This is a slightly different post to normal, in the sense its not really about papers but my take on agent-based modeling, urban analytics and the growth of Artificial Intelligence impacting both.
"In this episode of The Flux, host John Cordier sits down with Andrew Crooks ..... They dive into the world of agent-based modeling (ABM) - what it is, why it matters, and how it helps us simulate and better understand human behavior in complex systems. From simulating traffic jams to modeling social influence on vaccine uptake, Andrew shares how data, geography, and synthetic populations are revolutionizing our ability to forecast and inform decisions. They also explore the growing role of AI tools in democratizing modeling, the evolution of computational capabilities, and even ask: what if we had run a simulation before Brexit?"
Next up, I was asked to give a talk back in late May to give a seminar talk at the Department of Geography and Spatial Sciences (GSS) at the University at Delaware hosted by Yao Hu. The title of the talk was "Monitoring and Analyzing Cities through the Lens of Urban Analytics" In this talk I reflect what urban analytics means to me and how the field is changing. If this sounds of interest, below you can read the abstract to my talk and also see the recording. However, before ending this I would really like to thank Yao for hosting me, and the others from the GSS and the universty at large for making it a great visit and being an engaged audience.
Abstract:
For the first time in human history, more people are living in cities than rural areas and this trend is only expected to grow in the coming decades. This growth will place unprecedented challenges on cites with respect to sustainable development especially in light of climate change and increasing populations. One way to explore and understand cities is through the lens of urban analytics, a set of methods that allow us to monitor, analyze and model urban areas. This talk will explore how urban analytics has changed over time and showcase how our understanding of cities has benefited from it. I will showcase how new sources of data can be used to monitor and analyze cities and how in turn these can be integrated into models to explore various aspects of city life from pedestrian movement to urban growth. The talk will conclude with a discussion and demonstration of how artificial intelligence can be integrated into the urban analytics toolbox and what opportunities and challenges it poses.
Abstract: Urban areas now provide homes for more people than ever before, and with more and more people living in cities achieving sustainable cities is crucial for the betterment of all. Coinciding with the growth of the world’s population is the growth of artificial intelligence (AI) is which is becoming pervasive in all aspects of our daily lives. In this talk I will discuss how AI is offering us new opportunities when it come studying cities, specifically, through the lens of urban analytics. Urban analytics can be broadly defined a set of methods to explore, understand and predict the properties of cities. Through a series of examples, I will highlight how AI especially through the use of multimodal large language models (LLMs) is offering accessible methods for geographic information extraction and modeling of cities. I will showcase how AI can improve the granularity of urban data collection while at the same time provides more advanced GIS tools to practitioners in a more accessible and user-friendly way. However, AI alone is not the panacea when it comes to archiving urban sustainability and many challenges exist and the talk with conclude with these.
As such the paper demonstrates how social media data can act as supplementary source for dust events monitoring and captures the seasonal trends of such events. Furthermore, the paper highlights the potential of using crowdsourced data for the often overlooked field of dust monitoring that has substantial health and economic impacts.
If this sounds of interest, below we provide the abstract to the paper along with some figures which showcase our methodology and comparison with National Weather Service dust advisories and VIIRS satellite data. At the bottom of the post, you can find the full reference to the paper along with a link to it.
Abstract:
Dust storms and other dust events are natural phenomena characterized by strong winds carrying large amounts of fine particles which have significant environmental and human impacts. However, capturing the occurrence of such phenomena is a challenge. Previous studies have limitations due to available data, especially regarding short-lived, intense dust storms and events that are not captured by observing stations and satellite instruments. In recent years, the advent of social media platforms has provided a unique opportunity to access vast amounts of crowdsourced data. This paper explores the utilization of Flickr and X (Twitter) data to study dust event occurrences within the United States and their correlation with National Weather Service (NWS) advisories. The work ascertains the reliability of using crowdsourced data as a supplementary source for dust events monitoring. Our analysis of Flickr and X indicates that the Southwest region is most susceptible to dust events, with Arizona leading in the highest number of occurrences. On the other hand, the Great Plains show a scarcity of crowdsourced data related to dust events, which can be attributed to the sparsely populated nature of the region. Furthermore, seasonal analysis reveals that dust events are prevalent during the Summer months followed by Spring. These results are consistent with previous traditional studies that did not use social media of dust occurrences in the U.S., and Flickr-identified images of dust events show substantial co-occurrence with regions of NWS dust warnings. This paper highlights the potential of using crowdsourced data for the often overlooked field of dust monitoring that has substantial health and economic impacts.
Keywords: Dust storms, Crowdsourcing, Social media, Weather.
Flowchart of our workflow
Selected posts retrieved from X showing active dust events.
Selected images retrieved from Flickr showing active dust events.
Map showing the distribution of flickr-identified dust event occurrences, X-identified dust event occurrences, National Weather Service dust advisories, including dust storm (DS) warnings and blowing dust (DU) advisories.
Seasonal cycle of dust events using social media metadata, the National Weather Service advisories, and the VIIRS satellite data.
Examples of social media identified dust events and satellite observations for the same day. Brown shaded pixels indicate locations Suomi-VIIRS observed dust particles. Any VTEC warnings issued by NWS for the location are shown after the date of each dust event, with HWW and DSW indicating High Wind Warning and Dust Storm Warning, respectively.
Full Referece:
Adegbola, F., Crooks, A.T. and Evans, S.M. (2025). Crowdsourcing dust storms in the United States utilizing social media data. GeoJournal, 90(3), pp.1-18. Available at https://doi.org/10.1007/s10708-025-11359-9 (pdf)
If this sounds of interest, below you can read the abstract to our paper, see our workflow and resulting smellscape map. While the the analysis steps, along with the smell dictionary used, are documented in the research code compendium at https://figshare.com/s/8418d47cdc5c539b78ab. Finally at the bottom of the page, you can find the full reference and a link to the paper.
Abstract:
Smells can shape people’s perceptions of urban spaces, influencing how individuals relate themselves to the environment both physically and emotionally. Although the urban environment has long been conceived as a multisensory experience, research has mainly focused on the visual dimension, leaving smell largely understudied. This article aims to construct a flexible and efficient bottom-up framework for capturing and classifying perceived urban smells from individuals based on geosocial media data, thus, increasing our understanding of this relatively neglected sensory dimension in urban studies. We take New York City as a case study and decode perceived smells by teasing out specific smell-related indicator words through text mining techniques from a historical set of geosocial media data (i.e., Twitter/X). The data set consists of more than 56 million data points sent by more than 3.2 million users. The results demonstrate that this approach, which combines quantitative analysis with qualitative insights, can not only reveal “hidden” places with clear spatial smell patterns, but also capture elusive smells that might otherwise be overlooked. By making perceived smells measurable and visible, we can gain a more nuanced understanding of smellscapes and people’s sensory experiences within the urban environment. Overall, we hope our study opens up new possibilities for understanding urban spaces through an olfactory lens and, more broadly, multisensory urban experience research.
Key Words: geosocial media, multisensory urban experiences, network analysis, New York City, smellscape, text mining, urban smells.
A framework of deriving perceived smells.
An overview of research workflow.
An overview of the six dominant overlapping smells across New York City using the weaving mapping method. The weaving map uses the concept of strands to represent attributes. Each strand here represents one specific smell category, with the intensity of the color changing based on the density of that smell category within each neighborhood (i.e., grid cells).
Full Reference:
Chen, Q., Poorthuis A. and Crooks, A.T., (2025), Mapping the Invisible: Decoding Perceived Urban Smells through Geosocial Media in New York City, Annals of the American Association of Geographers. Available at https://doi.org/10.1080/24694452.2025.2485233. (pdf)
As the AAG has just wrapped up I thought I would write brief (well actually quite long) post on the talks that I was involved with at the conference. These talks would not have been possible without the many great students and colleagues who I have been collaborating with over time. Below you will find a brief summary of the talks and if any sound interesting, please reach out and we can give you more details.
First up (in order in which they were presented) was "Utilizing Streetview Images for Mapping Building Attributes with ChatGPT" with Qingqing Chen and Linda See. In this talk we discussed how multimodal Large Language Models are giving us a new way to study cities, in the sense, lowering the boundary for information extraction. Using ChatGPT andstreet view images from Mapillary as an example, we showed how one can extract building age, usage (e.g., commercial, mixed use, residential) and estimate building height which could all be used to inform urban climate models which require detailed information on buildings.
Abstract:
With increasing rates of urbanization, many challenges are emerging regarding sustainability such as the energy usage of buildings. Coinciding with this is the growing attention of urban climate models for energy demand estimation and climate adaptation strategies. However, the applicability of these models is constrained by the lack of detailed urban surface information. Therefore, creating comprehensive datasets that capture urban surface information at a granular scale is crucial for responding to our rapidly urbanizing world. Recent advancements in Large Language Models (LLMs) have opened new opportunities in urban studies, offering accessible methods for information extraction. In this talk we explore the feasibility of ChatGPT to extract building attributes from images. Taking New York City as a case study, we collect building images from Mapillary and process them through ChatGPT by posing specific questions to extract building attributes (e.g., height, functions, age). These attributes are then compared with authoritative data. The proposed method helps address the current dearth of fine-grained surface data on urban issues, therefore enhancing the accuracy and utility of urban climate models. Overall, this study demonstrates the practical applications of ChatGPT in geographic knowledge extraction, advancing the understanding of LLMs in geographic contexts, and more broadly to the discourse on Artificial Intelligence (AI) in urban modeling and climate science.
Keywords: Buildings, ChatGPT, Large Language Models (LLMs), Mapillary, Street View Images (SVI), GeoAI.
Example of Workflow.
Reference:
Chen, Q., See, L. and Crooks, A.T. (2025), Utilizing Streetview Images for Mapping Building Attributes with ChatGPT, The Association of American Geographers (AAG) Annual Meeting, 24th –28th March, Detroit, MI. (pdf)
This was followed by a talk by lead by Qingqing Chen entitled "Multi-sensory Experiences: The Connection Between the Smell and Vision in Understanding Urban Environments" where we explored to what extent can visual data from street view imagery be used as a proxy for capturing large-scale urban smell perceptions when compared to geosocial media. Such as what visual cues evoke specific smell perceptions.
Abstract:
Smell is a crucial transversal sense, which bridges the tangible aspects of urban environments, such as exhaust and garbage, with their intangible impacts on emotions, social interactions and well-being. Despite its crucial role in our everyday life, many urban studies primarily focus on the visual dimension, potentially introducing biases in our understanding of urban spaces. This research transcends this visual-centric bias by integrating the olfactory perceptions to investigate the nuanced relationship between smell and vision in urban environments. Specifically, we utilize advanced semantic segmentation to extract visual elements from street view imagery (i.e., Mapillay) and apply casual forest analysis to examine their causal effects on smell expectations recorded from human participants. These expectations, often tied to personal experiences and/or cultural associations, are compared with real-environment smell experiences derived from geosocial media (i.e., Twitter/X). The results show that visual cues can predict smells in straightforward urban settings, such as small parks or less densely populated areas. However, in complex urban environments, the predictive power of visual cues diminishes as diverse and overlapping scents obscure specific smells, even in visually distinct areas. These findings underscore the importance of a multisensory approach in urban studies, enhancing our understanding of the complex interplay between sensory experiences and informing urban design strategies that integrate multiple senses to create more engaging and inclusive environments. This is especially important for individuals with sensory impairments, such as anosmia or visual impairments, who rely on other senses to compensate for their perception of urban environments.
Keywords: Multi-sensory Experiences, Smell and Vision; Semantic Segmentation, Causal Effects, Geosocial Media, Street View Imagery (SVI).
Workflow
Reference:
Chen, Q. and Crooks, A.T. (2025), Multi-sensory Experiences: The Connection Between the Smell and Vision in Understanding Urban Environments, The Association of American Geographers (AAG) Annual Meeting, 24th –28th March, Detroit, MI. (pdf)
In the Geosimulation session that we organized, we had a talk entitled "Large Language Models for Conceptualizing, Designing, and Generating Agent-based Models" where Na Jiang, Boyu Wang and myself presented our work on exploring using multimodal Large Language Models (LLMs) to create age-based models. In the sense as modelers, we spend a lot of time developing and writing code and we were curious what could be done though the use of LLMs.
To give a sense of what is possible, below is an example of using ChatGPT for creating a model from a published paper.
Abstract:
Large language models (LLMs) play an important role in AI-powered code assistants such as code completion, debugging, and documentation. Such models can be further fine-tuned on smaller amount of data for specific tasks, often with the improvement of performance compared to generic LLMs. However, such fine-tuning techniques are seldomly used in generating sophisticated agent-based models (ABMs), because they are often implemented as software that demands extra standards such as the “Overview, Design concepts, and Details” (ODD) protocol. This research examines how we can bridge this gap by utilizing LLMs in designing or conceptualizing, building, and running agent-based models in the form of user prompts. . In this work, two models are created to demonstrate the proposed method. Specifically, Sakoda’s checkerboard model of social interaction is created by LLM from explicit design and description through prompts. The other model stimulates consumer preferences and restaurant visits as designed and implemented by a LLM. These models are evaluated by human experts on their code correctness and quality for both verification and validation purposes. This work serves as a first step towards fine-tuned LLMs on existing models and documentations to create high-quality and functional ABMs based on either user prompts or standard protocols, contributing to further exploration on the future of AI-assisted geospatial simulation development.
Keywords: Agent-Based Modeling, Large Language Models, Geospatial Simulation
Reference:
Jiang, N., Wang, B. and Crooks, A.T. (2025), Large Language Models for Conceptualizing, Designing, and Generating Agent-based Models, The Association of American Geographers (AAG) Annual Meeting, 24th –28th March, Detroit, MI. (pdf)
Next up was Ying Zhou who presented our work entitled "Identifying Environmental Characteristics That Influence Perceived Safety in Urban Spaces." In this work we explored how using social media data can be used to study the fear and how this relates to actual crimes within New York city. Broadly speaking we find through our analysis, that fear sentiment may spread out between the neighborhoods and their surrounding areas and that neighborhoods surrounded by crime-clusters may have high sentiments of fear.
Abstract:
One goal of creating livable cities is to enhance health and safety. While previous research in spatial analysis and urban planning has focused on correlations between physical environments and crime, typically relying on police-reported crime data from sources like the Crime Open Database (CODE), safety perception is inherently subjective and cannot be fully represented by objective crime statistics alone. Also, urban planning today has gradually shifted its focus from a top-down mechanism to a bottom-up mechanism, so understanding and fostering spaces where residents feel safe is essential. This research examines factors that contribute to residents’ perceived insecurity in New York City. In addition to spatial analysis of the open crime data, the research used social media data to acquire people’s perceptions. The result indicates that the aggregations of perceived unsafe locations overlapped with aggregations of crime data's locations, such as in Manhattan’s neighborhoods, but they do not overlap with each other entirely. By adopting Latent Dirichlet Allocation (LDA), a method of topic modeling, the research filtered and summarized the posted texts and contents related to the negative descriptions of places or spaces in the city, and then it identified the related characteristics of the environments. The characteristics are investigated by the method of local Moran’s I, which indicates their spatial autocorrelation in some neighborhoods in the city of New York. This research offers “bottom’s views” about urban safety for both urban planning and decision-makers, which contributes to people-centered consideration for future development and urban resource distribution.
Zhou, Y. and Crooks, A.T. (2025), Identifying Environmental Characteristics That Influence Perceived Safety in Urban Spaces, The Association of American Geographers (AAG) Annual Meeting, 24th –28th March, Detroit, MI. (pdf)
The last day of the conference was another busy day with two talks. First was entitled "PySGN: A Python Package for Constructing Synthetic Geo-social Networks" where Boyu Wang presented our work (with Taylor Anderson and Andreas Züfle) on a Python package that can be used to generate synthetic geo-social networks. As readers of this blog might know we have a an interest in social networks and using them in modeling and this package provides a toolkit for others to easily create their own geosocial networks (e.g., Geospatial Erdős-Rényi, Barabási–Albert and Watts-Strogatz models). For interested readers, the source code available at: https://github.com/wang-boyu/pysgn.
Abstract:
Synthetic population has been widely used in social simulations such as traffic modeling, pedestrian movements, and the spread of infectious diseases. In recent years, much attention was focused on generating synthetic population with social networks, that captures social connections between individuals. While synthetic populations are often geographically explicit, various algorithms have been proposed to create realistic geographic social (geo-social) networks, aiming to integrate spatial information into people’s social links. We build an open-source Python package, namely PySGN, for constructing synthetic geo-social networks that incorporates position information, exhibits small-world network properties, and can be scaled to hundreds of thousands and potentially millions of nodes. We discuss different ways of parametrizing the method, by either a global average node degree, or an expected degree for each individual node. It is demonstrated through a case study with synthetic population in Buffalo, NY. By doing so, we aim to illustrate how such synthetic geo-social networks can be created, utilized, and analyzed in downstream agent-based modeling and network analysis tasks. This work is available as an open-source Python package and integrated with the PyData ecosystem (e.g., GeoPandas, NetworkX), and can be further extended with more synthetic geo-social network algorithms in the future.
Wang, B., Crooks, A.T., Anderson, T. and Züfle, A. (2025), PySGN: A Python Package for Constructing Synthetic Geo-social Networks. The Association of American Geographers (AAG) Annual Meeting, 24th –28th March, Detroit, MI. (pdf)
The final talk (well for me) was presented by Fuzin Yin who presented our work with Lucie Laurian and Emmanuel Frimpong Boamah entitled "Analysis of Online Mutual Aid Network during Buffalo Blizzard 2022: Actors and Weak Ties." In this work we explored what kind of support was offered and requested over Facebook groups along with their network structures durring and shortly after the event utilizing machine learning.
Abstract:
In December 2022, Buffalo, NY encountered a once-in-a-generation blizzard that dropped over 4 feet of snow. This four-day snow event halted emergency services and left 47 dead. In the face of the devastating blizzard, Buffalonian demonstrated resilience and solidarity by establishing Facebook (FB) groups to share information and coordinate behaviors including donations, wellness checks, and snow removals. These spontaneous behaviors created an essential layer of protection when the major infrastructure was down. This research has collected data from Buffalo blizzard FB groups to analyze community-led self-help behaviors. We have used machine learning to classify FB messages into four categories (e.g., requesting help, offering help, emotional support, and other), and social network analysis to explore users’ communication patterns. Results show that out of all messages (n=9,988), 37% of them express emotional support, which is followed by messages offering help (25%). While requests for help constitute a small proportion (8%), they stimulate more replies than other categories. Network statistics suggest that the mutual aid network is low-density but with a high clustering coefficient. This implies that most group members are strangers with weak ties, but their connections are in the right place to allow efficient communication. However, users do not equally benefit where people requesting or offering help are central in online conversation while pure emotional supporters are at the periphery. We conclude that during the Buffalo blizzard 2022, online interactions translate into offline mutual assistance by establishing weak ties among disconnected users to facilitate the flow of information and resources.
Keywords: crisis informatics, mutual aid, social network analysis, machine learning, social media
Results of Mutual Aid Network during Buffalo Blizzard 2022
Reference:
Yin, F., Laurian, L., Crooks, A.T. and Boamah, E.F. (2025), Analysis of Online Mutual Aid Network during Buffalo Blizzard 2022: Actors and Weak Ties, The Association of American Geographers (AAG) Annual Meeting, 24th –28th March, Detroit, MI. (pdf)
While this is a rather longer post than normal, we hope you found it interesting and also as noted at the top of the post, if any of these talks/topics are of interest to you please feel free to reach out.
