Despite the trend towards widespread adoption of cloud-native formats and standards, much of the climate data publicly disseminated by NOAA has traditionally remained out of reach of people who could benefit most from it. Often, these potential users lack the necessary GIS and software engineering expertise to fully take advantage of the data. This includes everyone from scientists, to engineers, to urban planners, and even interested laypersons. But things have started to change. It is becoming increasingly obvious that pairing large language models (LLMs) with the right data and tools has the potential to bridge this gap, but how to do this well remains an open question.
It is exactly this question that we explored as part of our work on the NOAA-funded “Study to Determine Natural Language Processing Capabilities with the NCCF Open Knowledge Mesh (KM/NLP)” with a goal to build a conversational interface for asking arbitrary questions about NOAA’s climate data. We also wrote about this work earlier this year; with the project now concluded, we present a final highlight reel and share some takeaways.
The Foundations: Cloud-native Data Formats
If one wants to build an application that allows performing arbitrary analyses on real data, it is crucial to have that data stored in the most analysis-ready format possible. We started off this project experimenting with various ways to use NetCDF files directly, but found that to be too slow. We then transitioned Kerchunk-based virtual Zarr stores, and eventually to IceChunk stores. This required us to build a data ingestion pipeline that takes in NetCDF files from NOAA’s S3 buckets and builds virtual Zarr/IceChunk stores that reference those files. In the end, we ended up with 176 data variables across 36 distinct climate and weather datasets, which included some 3rd party Zarr stores published by dynamical.org.
An Agentic Harness Engineered for Trust
Our overarching goal was to produce reliable, consistent answers grounded in real data, and we designed each component of the workflow with that in mind.
Data Search Powered by a Knowledge Graph
To find the datasets most relevant to a given question, we use an agent to query a knowledge graph populated from metadata extracted from the original NetCDF files. This not only ensures that the search results are based on authoritative metadata, but it also speeds up searches. Because the results are presented to the agent in an easily digestible format, the need to access files at runtime or perform extensive parsing is precluded.

Data Analysis that is Scientifically Sound
For data analysis, on top of producing answers from real data, we wanted to ensure that the steps performed to arrive at any results were scientifically valid. A major design goal was to use, as much as possible, predefined “recipes” that had been written, reviewed, and validated by humans. This meant that letting the agent write and execute raw code was not the ideal approach.
The solution we arrived at was a compromise between the flexibility of the code-generation approach and the reliability and safety of the predefined-recipes approach. We call it Constrained Compositional Synthesis. Under this scheme, the agent can only use predefined recipes but is allowed to compose them together to represent complex multi-step algorithms. A rich enough function library (the collection of recipes), allows the agent to fulfill a very wide variety of data analysis requests.

A Multi-Agent Setup
We distribute the responsibilities across three agents: a Data Search Agent and a Data Analysis Agent (AKA the Algorithm Agent) as discussed above, plus an Orchestration Agent that interacts with the user and delegates tasks to the other two. This separation of concerns allows us to keep each agent’s context clear of irrelevant data. It also allows us to evaluate and optimize each agent in isolation.

Context Engineering, Structured Outputs, and the Probabilistic-Deterministic Interface
One of the design principles we stuck to was to move the fixed and predictable parts of the functionality to deterministic code as much as possible and to only leave the truly “magical” bits (such as understanding natural language and performing complex reasoning) to LLMs. This informed implementation choices such as:
- A pass-by-reference mechanism whereby large data (such as raw GeoJSON) is automatically uploaded to cloud storage and the agents receive a reference to it (along with summarized metadata) that they can then pass around to other tools without getting their context overwhelmed. This also sidesteps the common pitfall, seen in naive implementations, of LLMs having to redundantly transcribe large amounts of data from one tool to the next, which is slow, error prone, and wasteful in terms of tokens.
- Structured inputs and outputs with strict schemas and validation logic via Pydantic. All agents take in structured outputs as inputs and (with the exception of the Orchestration Agent) produce structured objects or tool calls as outputs. Even agent-to-agent communication–i.e., when the Orchestration Agent delegates tasks to other agents–is mediated through tool calls with strictly defined inputs and outputs. All of this ensures that we know the shape of the data moving through the system at any given time and provides a predictable interface between its probabilistic and deterministic parts.
AI Evals
When integrating LLMs into any system, it is important to thoroughly evaluate them against an evaluation dataset. This provides an objective measure of their performance that one can use to set expectations for the real-world performance and also to choose the right model for the job.
But while evaluations are important, they are also hard to do well. Challenges include: the open-ended nature of the inputs and outputs (even if structured), the inherent non-determinism in the outputs (even with temperature = 0), and the need to evaluate the LLM and the harness around it as a system rather than just the raw model.
We put a lot of effort into designing extensive evaluation datasets and used them to benchmark different configurations for the Data Analysis and Data Search agents and choose the optimal ones.


Looking Forward
This application was intended as a prototype and we do not intend to open it to the public. Moving forward, however, we intend to subsume this work into our new Limner offering. Limner is a similar but even more ambitious undertaking that provides a conversational, agentic interface for not just climate data but all kinds of data–both geospatial and beyond.
If you’re interested in discussing this work, or Limner, or learning more about querying and analyzing geospatial data more generally, we’d love to connect! Reach out to our team on our contact page any time.
Acknowledgements
This work is a collaboration between Element 84 and TechTraverse as part of the NOAA-funded “Study to Determine Natural Language Processing Capabilities with the NCCF Open Knowledge Mesh” BAA. Project contributors include: Kendall Aubertot, Sean Malone, Jason Gilman, Evan McQuinn, and Stephanie Wall.

