David Gasquez

@davidgasquez.com

5361 Followers 1336 Following

Data @ Protocol Labs. Open Data, Open Source, Open Protocols. Walks taker. Progressive Metal enjoyer. davidgasquez.com

When I work with agents, my first instinct is usually to “just prompt” through the task. That is fine for many coding-adjacent tasks, but not everything I do can be easily codified, and I bet the same thing happens to you! I’ve explored how to make agents do subjective work better, and I think there is another underrated approach to using agents for subjective tasks, or tasks where the goals are fuzzy and you’d like to keep the final call: build local and ephemeral workbenches. The idea is to have agents produce small pages/apps for the task at hand. For me, the clearest recent example is a grant review workbench I built. Instead of asking a model “which application is better?”, I got it to build me a workbench to help me better understand the task. In the workbench, I could see and sort every application, add random tags or notes as I read through them, visualize them on a 2D map to spot duplicates or core applications, etc. The useful part was the handoff. I could click a button to copy the state and ask my agent to do things like complete the labeling from my manual labels, recluster, add a new field to applications, suggest due diligence questions, etc. This gave both of us a high-bandwidth way to share state and made the feedback loop faster than chat. Chat alone is a bad interface for this kind of work because too much state stays implicit, or even hidden from you. With the workbench, I could inspect both the applications and the changes the agent wanted to make.
Complexity is the mind killer TL;DR: When faced with a choice, always pick the simplest thing that solves your immediate problem. Then make sure you can change your mind later. By the simplest thing I mean the solution that is easiest to reason about. When we always pick the simplest solution we avoid the analysis paralysis of finding out which solution might be the best in all the future scenarios we can imagine. We simply pick the simplest thing, and move on, knowing that it is easy to change our mind later. After we implement the simple solution, we have a much better understanding of our problem and we can ask "is my immediate problem solved?" If not; repeat. Complexity is the mind killer. Our mind gets overloaded when we try to change a complex system, and this causes us to make mistakes. Systems must be changeable, it is what keeps them from becoming legacy and it keeps us agile and adaptable. A lot of complexity gets added in the name of changeability. We create microservice systems so we can change our programming language and service implementation, but we add the complexity of network, service discovery, message queues, container orchestration and so on. It also adds a resource overhead that forces us to scale horizontally earlier, which is complex. Could we simply have hidden the implementation behind an interface or a function? Will we actually change our programming language any time soon? Does it actually solve our immediate problem?
Two stick figures talk. Caption: “A weighted random number generator just produced a new batch of numbers.” One says, “Let’s use them to build narratives!” Bottom caption: “All sports commentary.”
A web visualization titled “davidgasquez.com follow map”. It shows a large network map of followed accounts, represented by circular profile images. Accounts are grouped into colored clusters with labels such as: - Software engineering writers - Data Science AI Practitioners - R statistics community - Tech internet figures - Spanish data journalism - Data visualization cartography - Data Engineering and Analytics - Distributed data systems - Python data science OSS - Open source data tools - AI research community - AI and ATProto builders - Decentralized protocol builders On the right, a legend lists each cluster with a color and count. The largest groups appear to be AI and ATProto builders, Data Engineering and Analytics, Python data science OSS, and AI research community. There is also a size scale in the bottom right indicating follower count, from “fewer” up to about 202,723 followers. A Reset View button appears near the top left.
all my posts as they appear on davidgasquez.com
1 mu newsFeedPrefs🔥 social.mu.newsFeedPrefs 221 active +Infinity% 2 certified profile🔥 app.certified.actor.profile 151 active +Infinity% 3 hypercerts attachment🔥 org.hypercerts.context.attachment 133 active +Infinity% 4 hypercerts collection🔥 org.hypercerts.collection 130 active +Infinity% 5 certified organization🔥 app.certified.actor.organization 126 active +Infinity% 6 hypercerts activity🔥 org.hypercerts.claim.activity 116 active +Infinity%
Screenshot of a dark-themed code editor with a YAML index file open beside a Markdown spec titled “Atmospheric Data Portals.”
Screenshot of a tweet by Julien Chaumond saying Hugging Face is becoming bullish on data infrastructure. The tweet says he cloned 68 TB to a Hugging Face training bucket in under two minutes using Xet deduplication and infrastructure optimizations. Below the text is a video still of a Hugging Face “Copy to bucket” dialog showing a copy in progress from jasperai/monet to julien-c/my-training-bucket/monet-copy, with 178+ files, total size 68.2 TB, and an estimated time of less than one minute.
You’ve probably heard this already but, I keep coming across this pattern and I wanted to add another post to the cause. Generating a wall of text is now free while reading, verifying, and distilling still cost time and effort on the recipient. That asymmetry is what makes you sharing that document consisting of raw unrequested AI output rude. As this vibecoded website says, stop sloppypasta! The overall principle I keep in mind is what you send should take you more effort to produce than it takes me to read. This applies to chats, but also to code, bug reports, PRs, emails, docs, and now, it seems slides too. This is also especially bad when the point of the writing is to demonstrate your thinking. Rewriting that with an LLM changes meaning subtly, blurs authorship, and erodes voice. And people can tell. There is also “good uses of slop” though! I think it is ok to: Send a draft you’ve read, edited, and would defend as yours. Share the output as a side artifact. I like to see other folks’ raw prompts and sessions (I’ve learned a lot from reading Simon Willison prompts) so I can learn from it or fork it and tweak it. So, two small asks: Keep slop to yourself and share drafts once you’ve read it, verified it, and distilled them down to what actually matters. If you do share it, disclose it. Ideally send the prompt and a link to the chat rather than the output. Sessions are more interesting than transcripts, and the recipient can tweak and run their own models on top of it (I don’t usually trust people’s LLM context management skills). If your prompt is too embarrassing to share, that’s a signal worth listening to. These new manners are still emerging and I’m probably also offending folks with this sometimes (slopiness is an spectrum). But, let’s at least try not to outsource our thinking onto each other. Otherwise, we all end up drowning in slop or delegating to agents to get more slop!
My atmosphere data lives at at://davidgasquez.com. You can explore it all without API keys, auth, or arbitrary HTML to parse. Your agent can browse it, query it, and link to it. Getting someone latest posts is one click/curl away. That’s the pitch. The rest of this post is why it works, and why it might be the substrate your agents have been waiting for. Hostile Platforms You’ve probably experienced more than once recently, platforms being hostile to you (or your agent) getting the data out: throttled or limited APIs, bans for scraping, no identity persistence, no real-time access, walled gardens. Every integration is a deal that can be revoked when the CEO wakes up in a bad mood. The AT Protocol is an amazing technology that I think is very underrated for agents! Turns out, the properties Bluesky (the biggest AT Protocol application as of today) needed for humans (portable identity, open data, federated infrastructure, structured schemas) happen to be exactly what you’d want for agents too. As a quick and dirty intro (you should read the official ones, or this great walkthrough by mackuba), in the atproto world, every user is a personal, signed JSON repository. Every record (a post, a like, a follow, a photo, a blog post, anything really) is addressable by an at:// URI, has a public schema (a Lexicon), and is broadcast in real time on a global event stream (the firehose). Dan Abramov explains all of this in more detail and clarity in “A Social Filesystem”.
SKILL file
It has been a couple of months since I wrote about Barefoot Data Platforms. Since then, I’ve been applying those ideas into a real data platform I’ve been rebuilding from scratch. Learned a lot about what makes these platforms work well, especially with agents. Building It As a quick overview, the entire orchestration engine (fdp) is ~700 lines of Python. Assets are plain .sql or .py files with ugly but useful metadata in comments. No decorators, no registration, no config files. You can drop a file in assets/, and the next run picks it up, resolves dependencies, and materializes it! Delete the file, and it’s gone too. This minimalist, low-abstraction and filesystem based conventions data platform is great fit for agents:
Agentic engineering is mostly about building reliable systems around unreliable components (like your friendly coding agent). A good analogy I like is how early computers were powerful, but not trustworthy enough to be used “raw”. Hardware failed, bits flipped, and storage was noisy. Engineers had to build systems around the machine to make things more predictable. And, they came up with a bunch of interesting ideas! Error correction codes Redundancy Checksums Validation layers Retry logic With these, even though reliability was not a property of the machine/computer, it became a property of the system. This is something close to where we are with “Agentic Engineering”. As you’ve probably experienced, a coding agent can be fluent, useful, and very wrong at the same time! The key is to, like the early programmers did, treat agents as noisy components instead of trying to get the perfect/bigger one.
Terminal running through the tool examples.
A person in flowing white, sheer fabric stands on a stone balcony inside a large circular architectural space, their gauzy garments billowing dramatically as if caught by wind. Behind them, a tall narrow doorway opens into darkness, framed by concentric textured patterns that emphasize the scale and symmetry of the structure.
I’ve written about specializing Codex and Claude Code in the past. Here is how to do something similar for Pi! I made a small profile extension so I can launch somewhat contained agents with pi --profile <name>.1 A Pi profile in this case is just a small YAML file with some keys like model, thinking level, system prompt, and an allowlist of skills. model: openai-codex/gpt-5.4 thinking: medium system: You are SAM, a pragmatic assistant. Concise and useful. skills: - todoist-cli - agent-browser When Pi starts, the extension loads that profile, switches to the configured model, replaces the system prompt for the turn, limits which /skill:* commands are allowed, and keeps sessions isolated per profile.
An agent runs tools in a loop to achieve a goal. Agents are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. Agents help decouple programming, the craft of physically typing code, from engineering, the architecture of your system, the goals, the “why” of what you’re building. Using LLMs for coding is difficult and unintuitive, requiring significant effort to master. Don’t delegate thinking, delegate work. Before coding, make the plan with the model. You can use the same or a different model to critique the plan and iterate. If you are unsure, ask to “give a few options before making changes”. Redoing work is extremely cheap. Prioritize exploration over execution (at first). Iterate towards precision during the brainstorming phase. Start fresh once you know what and how to build it. Failed attempts are cheap. If the plan fails and the result is bad, just delete everything and try again. Divide the problem into smaller problems (functions, classes, …) and solve them one by one. Keep sessions short. Use Progressive Disclosure to ensure that the agent only sees tasks or project-specific instructions when it needs them.
Text of the blog post
embedding of all mercadonas
fiiles
CXDB - AI Context Store CXDB is an AI Context Store for agents and LLMs, providing fast, branch-friendly storage for conversation histories and tool outputs with content-addressed deduplication. Built on a Turn DAG + Blob CAS architecture, CXDB gives you: Branch-from-any-turn: Fork conversations at any point without copying history Fast append: Optimized for the 99% case - appending new turns Content deduplication: Identical payloads stored once via BLAKE3 hashing Type-safe projections: Msgpack storage with typed JSON views for UIs Built-in UI: React frontend with turn visualization and custom renderers
-- asset.name = filtered_numbers -- asset.schema = raw -- asset.description = Filtered values from transformed numbers -- asset.depends = raw.transformed_numbers select value, square, label, double, parity from raw.transformed_numbers where value >= 3
So, how does a minimal and opinionated data platform look like? Minimalism for me here means low abstraction and no frameworks. You write your scripts. These scripts are assets, and these assets depend on each other 1. That’s it. The opinion side of things is the interesting part. Here are the key ideas of the Barefoot Data Platform I built: Classic functional, declarative, independent, composable, and idempotent transformations Co-located assets, metadata, and documentation so assets have everything needed to be understood and reproduced in the same file One storage layer. DuckDB or SQLite, Parquet files, or whatever you want as long as you define a materialize function Many automated checks (ruff with all the rules enabled, ty, data tests, prek hooks, custom bdp check, …) for the agents
uvx.sh Install Python tools with a single command. Powered by uv. For example, to install ruff:
Company Knowledge Management All Organizations produce some kind of knowledge. If not properly managed, it’ll lie on your ex-employees, oral history and tribal knowledge. If we think about a company as an organism, then a knowledge management system is essentially the (collective) brain that keeps that organism alive and running. A corporate knowledge management system, ideally, contains every single bit of codifiable information within the company resulting in a library of all projects, processes and procedures. Managing an organization’s knowledge is mainly a people problem, not a technology problem. If the Culture is not properly set up, no one will do it and no amount of technology is going to help you magic it out of the ether. --- All monospace.
’ve been using Claude Code to take care of scrappy data cleaning tasks for a while. These days though, I’m using Codex as my coding agent. Similar to what I did with Claude, I’ve been “fine-tuning” Codex CLI to work on a few different vaguely defined tasks like classification, voting, filtering, or ranking. The pattern in this post works surprisingly well when you have the following conditions: Loosely defined open-ended tasks. e.g., tagging tweets with a set of predefined labels, extracting structured information from a GitHub issue, … Powerful agentic capabilities. Doing the task requires something more than a simple llm call or PydanticAI script. e.g., using gh api CLI to get the number of stars of a repository. Structured outputs. You need a response in a certain shape! This is something codex exec can do that claude couldn’t and is really powerful. e.g., return exactly True or False and nothing else. Save money! Unlike llm or other tools/libraries that require an OPENAI_API_KEY, Codex can use your ChatGPT subscription, making things “free”.
Data Culture The data team needs to be focus on delivering insights and supporting decisions. The outcome of the data team are decisions and a shared context across the organization that makes coordination easier. Your goal as a data professional is to facilitate decision making and help surface/investigate the performance of a business (e.g. operational). Learning to drive decisions quickly, a bias to action, is a critical competency for an analyst. Every skill you learn – communication, writing, experimentation, metric design – supports this. If analysis is not actionable, it does not really matter. Analysis must drive to action. Clear results won’t spur action themselves. The organization needs to be ready to pivot when something isn’t working. Data doesn’t make decisions, people do. Data’s impact is tough to measure — it doesn’t always translate to value The value of “insights” is often unknown. The Data Team should be building and iterating the Data Product. Notebooks are a workshop. Production systems are the factory. Not everything needs to be put into production. Not everything should be a notebook. You need both. Lean in to the strength of each. Data is fundamentally a collaborative design process rather than a tool, an analysis, or even a product. Data works best when the entire feedback loop from idea to production is an iterative process. To get buy in, explain how the business could benefit from better data (e.g: more and better insights). Start small and show value. Run Purpose Meetings or Business Metrics Review. Purpose Meetings are 30 min meetings in which stakeholders, engineers and data align on the goal of a release and what is the best way to evaluate the impact and understand its success. Align on the goal, commit on metrics and design the data.
❯ curl -s "https://huggingface.co/datasets/datania/aemet/raw/main/valores-climatologicos/2025/07/16.json" | jq '.[] | select(.nombre == "GRANADA AEROPUERTO")' { "fecha": "2025-07-16", "indicativo": "5530E", "nombre": "GRANADA AEROPUERTO", "provincia": "GRANADA", "altitud": "560", "tmed": "30,3", "tmin": "18,8", "horatmin": "05:08", "tmax": "41,8", "horatmax": "16:09", "dir": "11", "velmedia": "2,2", "racha": "10,3", "horaracha": "16:30", "sol": "13,6", "presMax": "955,3", "horaPresMax": "Varias", "presMin": "951,0", "horaPresMin": "16", "hrMedia": "29", "hrMax": "60", "horaHrMax": "Varias", "hrMin": "13", "horaHrMin": "16:02" } ~
Screenshot of linkweaver running. It is downloading a few urls
a contest... ❯ uv run examples/energy.py 2025-07-05 17:15:20 - arbitron.agent - INFO - Initialized agent safety_specialist 2025-07-05 17:15:20 - arbitron.agent - INFO - Initialized agent environmental_specialist 2025-07-05 17:15:20 - arbitron.agent - INFO - Initialized agent economic_specialist 2025-07-05 17:15:20 - arbitron.contest - INFO - Starting competition 'Arbitron Competition' with 5 items and 3 agents 2025-07-05 17:15:20 - arbitron.contest - INFO - Agent safety_specialist will perform 10 comparisons 2025-07-05 17:15:21 - arbitron.agent - INFO - Agent safety_specialist chose nuclear_fission over wind_turbines 2025-07-05 17:15:23 - arbitron.agent - INFO - Agent safety_specialist environmental_specialist chose wind_turbines over nuclear_fission 2025-07-05 17:15:40 - arbitron.agent - INFO - Agent environmental_specialist chose nuclear_fission over hydroelectric_dams 2025-07-05 17:15:41 - arbitron.agent - INFO - Agent environmental_specialist chose geothermal_power over nuclear_fission 2025-07-05 17:15:43 - arbitron.agent - INFO - Agent environmental_specialist chose geothermal_power over hydroelectric_dams 2025-07-05 17:15:44 - arbitron.agent - INFO - Agent environmental_specialist chose solar_photovoltaic over nuclear_fission 2025-07-05 17:15:46 - arbitron.agent - INFO - Agent environmental_specialist chose wind_turbines over geothermal_power And you get a nice ranking! 1. solar_photovoltaic (score: 1.227) 2. wind_turbines (score: 1.127) 3. nuclear_fission (score: 1.127) 4. hydroelectric_dams (score: 0.801) 5. geothermal_power (score: 0.801)
Core Components Blips: Atomic units of information formatted as ATProto records, such as thoughts, concepts, and emotions Links: Typed relations between blips with semantic context Cominds: Specialized AI agents that process network activity Spheres: Workspaces defined by core directives that focus agent cognition Melds: Interaction protocol for accessing sphere knowledge
A screenshot with 3 windows. VSCode, btop, and Claude Code
import httpx import llm def read_url(url: str) -> str: """ Use Jina Reader to convert a URL to Markdown text. Example usage: llm -f 'reader:https://simonwillison.net/tags/jina/' ... """ url = "https://r.jina.ai/" + url response = httpx.get(url) if response.status_code != 200: raise ValueError(f"Failed to load fragment from {url}: {response.status_code}") return response.text source_note = llm.Fragment( content=""" # Documentation - If your product/tool/process documentation is not good enough, people will not use it. - Documentation changes should be reviewed in the same way as your code. - [If someone's having to read your docs, it's not "simple"](https://justsimply.dev/). Remove filler words. - [Principles to keep in mind when writing documentation](https://mkaz.blog/misc/notes-on-technical-writing/). - The purpose of technical writing is to help users accomplish tasks as quickly and effectively as possible. """ ) model = llm.get_model("gpt-4.1-mini") chain_response = model.chain( """ Where is "You’re keen to share your excitement" mentioned in the sources? """, tools=[read_url], # after_call=print, fragments=[source_note], system="You are a helpful research assistant. ALWAYS read ALL the links with `read_url`. Once you've readed all the links in the note, help the user with their questions.", ) for chunk in chain_response: print(chunk, end="", flush=True)
❯ lw Emergence.md | llm "Summary on one sentence" Emergence describes the unpredictable, complex outcomes that arise from simple interactions, which, while not top-down plannable, can be provoked by designing systems with adaptable building blocks, multiple pathways to goals, a low barrier to entry, and multi-agent participation.
Giving them proper context. Using simple and specific prompts. Describing small problems with clarity and boundaries. Making things easy to run and test (Makefile, Docker, …). Using well-known frameworks that appear in their training data. With these basic ideas in mind, let’s see what we can do to make the most out of the current LLMs’ capabilities.
❯ llm -m github-copilot/claude-sonnet-4 "Hello :)" Hello! 👋 Great to meet you! I'm GitHub Copilot, and I'm here to help you with any programming questions, code reviews, debugging, or development tasks you might have. Whether you're working on: - Writing new code - Debugging existing code - Learning a new programming language - Discussing software architecture - Code optimization - Or anything else development-related I'm ready to assist! What are you working on today?
DuckDB WASM shell. duckdb> select count(*) from 'https://huggingface.co/datasets/davidgasquez/ine/resolve/main/tablas/15797/datos.parquet';
Xet is an experimental feature that allows you to store large files efficiently. Xet is currently disabled.
Github Action showing a DuckDB run
White screen on Brave with the Network tab open
Screenshot of a section of the text with the following content hightlighted. my specialty is building systems that move large amounts of data through companies, organize them in a way that is at least marginally less of a horrific clusterfuck than what random people without specific training will do when left to their own devices, and sometimes assist with statistics
pydantic.run homepage wiht a simple polars example
README of the project showing a set of opinionated tools, code, and resources for Machine Learning projects, datathons, or ad-hoc competitions.
Table with data about random orders