A Dual-LLM Supervised Workflow for Replicating Agent-based Models

In the past we have written about large language models (LLMs) and how these can be used in agent-based modeling. However, these previous posts only touched the surface on what is possible. One area we are currently exploring is how LLMs can aid in model replication, which is a major challenge in agent-based modeling. In the sense, there are countless agent-based models but very few are replicated and if  a model is to withstand the test of time, replication is needed. 

To this end, Boyu Wang, JoAnn Lee and myself have an extended abstract entitled: "A Dual-LLM Supervised Workflow for Replicating Agent-based Models: From NetLogo to Mesa" at the 2026 Social Simulation Conference. In this work we demonstrate how LLMs can be used to replicate an exiting model into another modeling package, Specifically, we take the Ya-TASERPS model which was initially implemented in NetLogo and re-implement it in Mesa via a  LLM workflow. 

The workflow uses two GPT-5.4 models with distinct LLM roles under continuous human mediation. One LLM role handles planning and evaluation, from checking source code, decomposing tasks into phases, and reviewing whether implementations are consistent with the source NetLogo model. The other LLM role performs the actual implementations in phases, as outlined by the first LLM role.

Each task phase defines acceptance criteria, proceeds through implementation and verification, and ends with human review before progression. The human researcher mediates both LLM roles, resolves ambiguities, and decides whether the resulting model state is acceptable. The aim is not towards full automation but to reduce prompt drift, limit uncontrolled rewrites, and keep generated code subordinate to explicit validation. 

If you want to read about our replication, why we chose the Ya-TASERPS model along with our findings, please feel free to read the paper and more information about this can be found at: https://github.com/wang-boyu/Ya-TASERPS.

The NetLogo-to-Mesa replication workflow utilizing two distinct LLM roles.

Large-run prosocial outcomes across the six user-controlled inputs in NetLogo and Mesa with the same set of 729 parameter combinations, 10 random seeds, and 1800 days per run. The two implementations preserve the same broad significance and directional patterns.

Full Reference: 

Wang, B., Lee, J. and Crooks, A.T.  (2026), A Dual-LLM Supervised Workflow for Replicating Agent-based Models: From NetLogo to Mesa. Social Simulation Conference 2026, Durham, UK. (pdf)

New Paper: Online Interactions, Mutual Assistance and the Power of Weak Ties

In the past we have written about disasters and how one can model peoples reactions to them or how one can mine social media or mobility data to explore people's responses to them. In a new paper published in the Annals of the American Association of Geographers, Fuzhen Yin, Lucie Laurian and Emmanuel Boamah and myself continue this line of research. Specifically we explore how people exchanged resources and coordinated mutual aid via Facebook during the 2022 Buffalo Blizzard.

The paper itself is entitled "Surviving the Buffalo Blizzard: Online Interactions, Mutual Assistance and the Power of Weak Ties" In the paper we describe how we manually collected blizzard-related conversations from two Facebook groups "Buffalo S.T.O.R.M" and "Buffalo Blizzard". After which we utilize machine-learning (e.g., support vector machines) to identify mutual-aid messages which were then categorized them into four groups: aid requests, aid offers, emotional support and other. From which we then constructed a social network of users interactions during the blizzard to identify aid requests, aid offers, and emotional support messages during a time of crisis. As such our study contributes to the growing literature on human dynamics by examining spontaneous mutual aid during the blizzard and highlights how online mutual assistance operated through ‘phygital’ (physical–digital) integration.

If this sounds of interest, and you wish to find out more with respect to our findings, below you can read the abstract to the paper, see some of the figures which describe our research methodology and results while at the bottom of the post you can find a link to the paper itself.

Abstract: 

In December 2022, Buffalo, New York, experienced a once-in-a-generation blizzard. The four-day lake-effect snow accompanied by storm-force winds knocked down power lines, halted emergency services in several towns and resulted in forty-seven fatalities of residents who lost heat and power or were trapped in the snow. In response to the storm, Buffalonians demonstrated strong solidarity through quickly self-organized Facebook groups to exchange resources and coordinate mutual aid. Our study examines the emergence of grassroots mutual assistance through online–offline interactions and its impact on resilience in the physical world. We manually collected blizzard-related conversations, used machine learning to identify mutual-aid messages, and applied social network analysis to examine users’ interactions. Our findings reveal that Facebook users delivered life-saving assistance through online conversations involving requesting and offering practical, informational, and emotional support. The Facebook blizzard communities developed networks of weak ties that expanded access to vital resources and facilitated the flow of information and materials among disconnected residents. This research highlights virtual spaces as digital urban commons where strangers can benefit from emerging social capital during crises. It also offers insights for emergency management agencies seeking collaborations with grassroots online communities to develop formal–informal mutual aid strategies for future crises.

Keywords: Mutual aid, winter storm/blizzard, crisis informatics, weak ties, social network analysis, machine learning, social media.

Diagram of analysis workflow.

Geographical distribution of places mentioned in the Blizzard Facebook groups. (a) Kernel density map based on precise point locations. (b) General areas at three spatial levels: neighborhoods in Buffalo (purple), cities and towns (blue), and county subdivisions (green). The orange dashed line delineates the boundary of the kernel density map. Line widths and label sizes are proportional to each area’s prevalence in the online discussions.

Classifying mutual aid messages into four categories: request for support, aid offers, emotional support and other (n=9,599).

Tripartite message network capturing the information flow from posts to comments, from comments to replies, and within replies.

Social network of mutual aid interactions. (1) Users’ degree centrality with a log-transformed x-axis. (2) Users’ betweenness centrality with a log-transformed x-axis. (3) Social networks of users’ online interactions, highlighting four clusters: A, B, C, D. (4) Size distribution of detected communities. (5) Users’ composition in detected communities.

Full Reference: 

Yin, F., Laurian, L., Crooks, A.T. and Boamah, E. (2026), Surviving the Buffalo Blizzard: Online Interactions, Mutual Assistance, and the Power of Weak Ties, Annals of the American Association of Geographers. https://doi.org/10.1080/24694452.2026.2707171 (pdf)

Evaluating the Feasibility of ChatGPT for Mapping Building Attributes

In the past we have written about using Multimodal Large Language Models (MLLMs)  like  ChatGPT for coding of models and also  analyzing images. One advantage we see for MLLMs is that unlike traditional approaches that require extensive expertise in computational analysis, such as computer vision and deep learning, MLLMs leverage pre-trained capabilities that simplify the analytical process. This accessibility enables a larger group of researchers to incorporate MLLMs in their analyses when it comes to studying the form and function of cities at scale. To this end, we (Qingqing Chen, Linda See and myself) have new book chapter entitled "Evaluating the Feasibility of ChatGPT for Mapping Building Attributes" published in the open access book: "Geography According to Foundation Models" edited by  Krzysztof Janowicz, Rui Zhu, GengchenMai, Song Gao. Yingjie Hu, Zhangyu Wang, Ling Cai and Lauren Bennett.

In this chapter we evaluate the potential of MLLMs, in our case ChatGPT, to extract building attributes (e.g., age, use and height) from Mapillary street view images.  We find that ChatGPT was good at extracting some information and less good in other cases. For example it identified correctly 87% of the residential buildings. We also discuss ways to improve the results (e.g., using higher quality street view images, altering and refining the prompts). If you wish to find out more about our findings we encourage you to read the chapter. To give you a better sense of this research, below we provide the abstract to the paper, our case study area along with our workflow and a sample of the results. Finally at the bottom of the post, you can find the full reference to the chapter along with a link to it. 

 

Abstract:

With increasing rates of urbanization, many challenges are emerging regarding urban sustainability such as the energy usage of buildings. Coinciding with this is the growing attention of urban climate models for energy demand estimation and climate adaptation strategies. However, the applicability of these models is constrained by the lack of detailed urban surface information. Therefore, creating comprehensive datasets that capture urban surface information at a granular scale is crucial for responding to our rapidly urbanizing world. Recent advancements in Multimodal Large Language Model (MLLMs) have opened new opportunities in urban studies, offering accessible methods for information extraction. In this chapter we explore the feasibility of ChatGPT to extract building attributes from images. Taking New York City as a case study, we collect building images from Street View Imagery and process them through ChatGPT by posing specific questions to extract building attributes (e.g., height, functions, age). These attributes are then compared with authoritative data. The proposed method helps address the current dearth of fine-grained surface data on urban issues, therefore enhancing the accuracy and utility of urban climate models. Overall, this study demonstrates the practical applications of ChatGPT in geographic knowledge extraction, advancing the understanding of MLLMs in geographic contexts, and more broadly to the discourse on Artificial Intelligence (AI) in urban modeling and climate science.

The spatial distribution of Mapillary images within the study area, shown on the left, and the distribution of images by variance showing increasing image quality on the right.

An overview of the research workflow.

Comparison of the building period of construction from the ground truth data and the classifications from ChatGPT. (a) A confusion matrix which details the distribution of buildings classified within each period by ChatGPT compared to the ground truth data; (b) A chord diagram illustrating the patterns of agreement and confusion among the categories.

Comparison of building type classifications. (a) A confusion matrix detailing the distribution of ChatGPT’s classifications against the hand labels from experts; (b) A chord diagram illustrating the proportion of classifications for each building types as labeled by experts compared to ChatGPT’s classifications.

A comparison of building heights from ChatGPT and the NYC Open Data. (a) The correlation of height between the ground truth and ChatGPT; (b) The distribution of ground truth heights and the predicted heights; (c) The difference in the heights.

Full Reference:

Chen, Q., See, L. and Crooks, A.T. (2026), Evaluating the Feasibility of ChatGPT for Mapping Building Attributes,  in Janowicz, K., Zhu, R., Mai, G., Gao, S., Hu, Y., Wang, Z., Cai, L., and Bennett, L. (eds), Geography According to Foundation Models, IOS Press, Amsterdam, The Netherlands, pp. 107-120. (pdf)

New Paper: Exploring Fear in Urban Environments

In the past we have written about how we have used social media to study a plethora of topics with respect the the form and function of cities among many other things. But one thing we have not explored is fear and more specifically fear of crime and how this can be mined through geosocial media. 

This has now changed with a new paper entitled "Exploring Fear in Urban Environments: Place and Space Analysis of Social Media Data" which has recently been published in Applied Geography.  In this paper, Ying Zhou and myself extract fear related posts from social media and examine the places and spaces where people experience fear, as well as the factors that contribute to it in New York City. 

We do this by utilizing Natural Language Processing (NLP) techniques for sentiment and text analysis, including a RoBERTa-based emotion classification model and the BERTopic model for topic modeling. The former model narrowed the raw data to those with the dominant emotion of fear, and the latter analyzed space- and place-related features that contribute to the fear sentiment. Then, the selected social media data were analyzed using spatial clustering methods (i.e., Hotspot Analysis (Getis-Ord Gi*) and Local Moran’s I) and compared with urban crime data for weekly trends and spatial patterns. As such the paper has the following research objectives:

1. exploring places where people expressed fear through social media; 
2. making comparisons between safety-related fear and crime from the perspective of both time and space; 
3. extracting urban environmental and social features that lead to fear.

If this sounds of interest, and you wish to find out more with respect to our findings, below you can read the abstract to the paper, see some of the figures which describe our research methodology and results while at the bottom of the post you can find a link to the paper itself. Finally the code we utilized in the paper can be found at https://osf.io/y7xfc/overview.

Abstract:

One goal of creating livable cities is to enhance public safety. While previous research in urban studies has focused on correlations between physical environments and crime, it has typically relied on criminal statistics. However, fear of crime is an emotional response to perceived risks rather than a direct reflection of crime levels, so it cannot be analyzed solely by crime data. Additionally, urban planning today has gradually shifted its focus from a top-down to a bottom-up approach, making it essential to understand and foster spaces where residents feel safe. This research examines the spaces and places where people experience fear, as well as the factors that contribute to it, in New York City. We utilized social media data to gather people’s expressions of the city and identified posts expressing fear emotion using the RoBERTa-based model and a rule-based classifier. Then, the selected social media data and crime were compared temporally by weekly trends and spatially by clustering methods (i.e., Hotspot Analysis (Getis-Ord Gi*) and Local Moran’s I). The results show that their temporal and spatial patterns partially have limited alignment. To delve into the origins of fear, we extend our analysis by adopting BERTopic to identify topics and summarize them into themes (e.g., places, transportation, people, others) to understand the bottom-up emergence of fear, thereby informing a people-centered approach to research on urban issues. 

Keywords: Social media; Natural language processing; Sentiment analysis; Urban environment.

Methodology framework.

An example of textual analysis on fear-related tweets: from machine-generated topics to human-interpreted themes describing fear in NYC.

Weekly trends comparison between safety-related fear and violent crime.

Clustering features analysis by the method of hotspot analysis (Getis-Ord Gi∗).

Full Reference: 

Zhou, Y. and Crooks, A.T. (2026), Exploring Fear in Urban Environments: Place and Space Analysis of Social Media Data, Applied Geography, 192: 104051 (pdf)

New Paper: Connecting senses: The cross-modal associations between smell and vision in understanding urban environments

In a previous post we wrote about how one can mine social media to uncover smells and how they shapes peoples perceptions of urban spaces. Building off this work we (Qingqing Chen, Ate Poorthuis and myself) have a new paper entitled "Connecting senses: The cross-modal associations between smell and vision in understanding urban environments" published in Geographical Analysis. 

In this paper we build upon this but at the same time move away from social media to explore the relationship between smell and vision. We do this utilizing street view imagery from New York City, in this case we are using Mapillary. A subset of these images were labeled with pre-defined smell categories (e.g.,  ‘Nature’, ‘Food’, ‘Transportation & Fuel’) in order to develop a deep learning smell classifier capable of classifying perceived smells from the images. We then use  advanced image processing techniques (e.g., ResNet50, VGG16, Inception-V3, MobileNet and EfficientNet.) to extract visual cues from the street view imagery to predict smells. 

If this sounds of interest, below we provide an abstract to the paper along with some of the figures which show our workflow and accompanying results. At the bottom of the post you can see the full referece to the paper, while at https://figshare.com/s/94cdec3b14c206e6d225 we provide our code to allow others to replicate or extend this to other areas. 

Abstract:   

Smell is a crucial yet understudied sensory dimension in urban environments, bridging tangible elements (e.g., exhaust, flowers) with intangible impacts on emotions, social interactions and well-being. While geographical and urban research increasingly acknowledges multisensory experiences, much of geospatial analysis still emphasized the visual dimension. This research advances spatial thinking by examining cross-modal associations between smell and vision in urban environments. Specifically, we utilize advanced image processing techniques to extract visual cues from street view imagery (i.e., Mapillary) and apply causal analysis to examine their effects on smell expectations recorded from participants. The results show that visual cues can predict smells in straightforward urban settings (e.g., parks or less densely populated areas). However, in complex urban environments, the predictive power of visual cues diminishes as diverse and overlapping scents obscure specific smells, even in visually distinct areas. These findings underscore the importance of a multisensory approach in urban analytics, enhancing our understanding of the interplay between sensory experiences and informing urban design strategies that integrate multiple senses to create engaging and inclusive environments. This is especially important for individuals with sensory impairments, such as anosmia or visual impairments, who rely on other senses to compensate for their perception of urban environments. 

Keywords: Smell and Vision; Cross-modal Associations; Multisensory Experiences; Image Processing; Street View Imagery (SVI)

An overview of research workflow.

The visual feature extraction framework based on key patterns identified from questionnaires.

Spatial distribution of images with different perceived smells and example participant notes identifying potential smell sources responsible for perceiving the dominant smells.

Identified important features for smell categories that are relatively indirect to be inferred from visual cues. Left: Identified important features; Right: Examples of feature  contribution in individual images.

Full Referece: 

Chen, Q. Poorthuis, A. and Crooks A.T. (2026), Connecting senses: The cross-modal associations between smell and vision in understanding urban environments, Geographical Analysis, 58 (3): e70046.. Available at https://doi.org/10.1111/gean.70046. (pdf)