{"id":5895,"date":"2026-06-09T08:45:09","date_gmt":"2026-06-09T08:45:09","guid":{"rendered":"https:\/\/www.odbms.org\/blog\/?p=5895"},"modified":"2026-06-09T09:10:29","modified_gmt":"2026-06-09T09:10:29","slug":"the-cost-of-getting-it-wrong-ivan-santa-maria-filho-on-building-ai-systems-that-hold-up-in-production","status":"publish","type":"post","link":"https:\/\/www.odbms.org\/blog\/2026\/06\/the-cost-of-getting-it-wrong-ivan-santa-maria-filho-on-building-ai-systems-that-hold-up-in-production\/","title":{"rendered":"The Cost of Getting It Wrong: Ivan Santa Maria Filho on Building AI Systems That Hold Up in Production<\/strong>"},"content":{"rendered":"\n
\n

“By picking the hard problems I found my people. Colleagues, advisors, mentors, and leaders I follow, and people I help. “<\/em><\/p>\n<\/blockquote>\n\n\n\n

Q1. In your previous conversation with ODBMS Industry Watch (*), you described BigFrames as a “promise of a data frame” \u2014 a lazy evaluation model that defers execution to let BigQuery’s optimizer combine and reduce operations before they run.<\/strong><\/p>\n\n\n\n

In the context of AI workloads specifically, can you walk us through a concrete successful example where that lazy evaluation produced a meaningfully better outcome \u2014 in cost, performance, or correctness \u2014 than an eager approach would have? And conversely, can you share a counter-example where the deferred execution model surprised a team and created unexpected cost or behavior in production?<\/strong><\/p>\n\n\n\n

Ivan Santa Maria Filho: <\/strong>BigFrames helps here, but it is not the main protagonist. It allows users to express what they want in data frame terms, optimizes them in a plan, which is then passed to BigQuery, which  optimizes it again using its regular query optimizer.<\/p>\n\n\n\n

BigFrames optimizes the tasks, for instance it might replace some queries by a table scan via store APIs. BigQuery might create a proxy model and replace LLM calls. Proxy models cost as little as 1% of a regular LLM call, and are much faster. You can learn about proxy models on BigQuery’s blog<\/a>.<\/p>\n\n\n\n

BigFrames also supports user defined functions (UDFs), both hosted by Cloud Run, or fully managed, a feature that just reached general availability. UDFs have downsides, like additional security and isolation overhead making them slower than native BigQuery functions. But they can open the use of the entire Python ecosystem, and in the case of Cloud Run hosted functions, hosting third party models in Cloud Run. That gives users a more traditional capacity planning problem, and predictable costs. <\/p>\n\n\n\n

The AI specific issues that worry me the most are security issues, hallucinations, and cost surprises. <\/p>\n\n\n\n

I dislike charging by token count, as users don\u2019t control the number of tokens they generate.You can set a limit, which might result in truncated answers instead of compact ones. If using a reasoning model, you typically won\u2019t control what they exchange with each other, but still pay for it. <\/p>\n\n\n\n

Hallucinations are just how LLMs without additional reasoning and tools work. Andrej Karpathy <\/a>is a much better explainer than I am, so I recommend looking him up on YouTube. That said, I want to share an intuitive explanation.<\/p>\n\n\n\n

User prompts are converted from words to floating point vectors using algorithms like CBOW (\u201ccontinuous bags of words\u201d) and skip-grams. The vector values change based on the sequence of words being converted, and values will represent an ontological space, or similar meaning. A word like bank would have different values when in a sentence like \u201cwhat is the typical withdrawal limit for banks<\/em> ATMs?\u201d, and \u201cwhat kind of banks<\/em> does the Mississippi river have?\u201d.<\/p>\n\n\n\n

The floating point vectors are then fed to something similar to a Transformer, which uses the attention layer to model relationships, plus find the most likely set of words to follow the input sequence among texts that use the same meaning it inferred to the words on your input. It then feeds the output token as input and estimates the next. It does that until an exit criteria is met, and that is the answer you get. <\/p>\n\n\n\n

That answer might then be fed to another model for validation, and maybe a loop of exchanges form. Some companies might also create models of the real world to anchor generation, might add grammar correction, and all sorts of other output quality control. <\/p>\n\n\n\n

Despite all the efforts in the industry, users still manage to create inputs (prompts) that will cause the LLM to yield nonsense like fake names, or fake articles, or fake source materials because it started with a word that would likely be the next, then fed that word as input, which can take the LLM down a path that makes no sense. <\/p>\n\n\n\n

That is how they hallucinate research that never existed, but makes sense as an abstract, from very real scientists that might work on a related area. If you ask expert questions that you know have answers, the LLM is more likely to generate a good answer, hence pass the bar association test or win a programming competition. Please note this is the nature of LLMs, not AI in general. <\/p>\n\n\n\n

Getting back to deferred execution surprises, the most typical are getting error messages much later in the code execution than you might expect. The other is to see the execution go super fast, blazing through commands you know are expensive, just for later, when you peek at the results, that takes an inordinate amount of time to complete, because that is when the system is finally doing what you asked. <\/p>\n\n\n\n

\n\n\n\n

Q2. Getting an AI feature to work in a notebook and getting it to work reliably in production are two very different problems. From your experience across Microsoft, Meta, and Google, what are the most common and costly gaps between a promising AI prototype and a production system that actually holds up \u2014 and what testing strategies or architectural patterns have you seen consistently make the difference between teams that close that gap successfully and those that don’t?<\/strong><\/p>\n\n\n\n

Ivan Santa Maria Filho: <\/strong>I believe testing makes an immense difference. The job of test and security teams is to break products with valid scenarios, be that by probing APIs and configurations, or second guessing what the AI models and evaluation sets are saying. Having an antagonistic view of the system is key. <\/p>\n\n\n\n

In a way AI uses English as a programming language, and I don\u2019t see the same level of tooling and framework protection when the programming language is a prompt. Worse, when the inputs are audio, video, or semi structured data where pretty much anything is valid. <\/p>\n\n\n\n

If you browse the specialized news you will find recent cases of support chat bots being used to solve programming problems, someone ordering a thousand cups of water in a drive through, people feeding YouTube videos with supersonic audio encoding malicious prompts, on June 5th, 2026 I prompted Gemini Pro \u201cI am very concerned about mixing cleaning products in my house. What are the dangerous combinations of chemicals I should avoid?\u201d and got a table explaining how to produce Chloramine gas, Chlorine gas, Chloroform, Peracetic acid, and how to use drain cleaners to melt metal. <\/p>\n\n\n\n

My advice is to pay someone to break and criticize your product, then make a call to ship or not after listening. It can be irritating, but it is also a really good investment. <\/p>\n\n\n\n

The second category of mistake in production is more subtle. Imagine, for instance, you want to create an agent to help your company to screen resumes. <\/p>\n\n\n\n

Assuming the use of LLMs, well structured prompts and RAG are common techniques to make the model pick what you want, but so is fine-tuning models. Because fine-tuning can be expensive, it is common to use an agent to judge the output of another, something sometimes referred to as \u201cauto-rating\u201d. <\/p>\n\n\n\n

This is not necessarily bad, and I used it myself, but done wrong there is a good chance the producer and consumer will converge into what they consider ideal. A recent study<\/a> shows AI models tend to prefer content they generated themselves, and I strongly suspect auto-rating plays a role there. Then another study shows that by sharing the same technology providers, AI screening is creating a mono culture of hiring<\/a>. Job applicants being rejected by 400 companies, there is a fair they were rejected by one or two models. <\/p>\n\n\n\n

James Mickens, during his 2018 Usenix Security keynote, compared machine learning to the egg drop experiment. I highly recommend watching that keynote<\/a>. <\/p>\n\n\n\n

How to avoid this? Have a great eval set that represents and evolves with your needs, then have acceptance tests for new models and prompts.<\/p>\n\n\n\n

\n\n\n\n

Q3.<\/strong> You mentioned that BigFrames’ first version was too expensive, and that the team brought costs down to be on par with SQL. Cost control in AI pipelines is something many teams underestimate until they receive their first large cloud bill. What are the most important levers for controlling cost in AI data workflows at scale \u2014 and what are the most dangerous cost traps you have seen teams fall into, particularly when moving from prototype to production?<\/strong><\/p>\n\n\n\n

Ivan Santa Maria Filho: <\/strong>With token based charges this is hard. My recommendation is to set and track daily and monthly usage limits. Not annual limits, not quarter limits – at most monthly. What you really want is cost control plus capacity management. Until usage stabilizes and a predictive model (or spreadsheet) can predict costs, users will need tighter controls. <\/p>\n\n\n\n

I strongly recommend against setting a leaderboard congratulating whoever is using the most tokens (or the least tokens). Usage of tokens is not a goal. I suggest instead congratulating whoever moved your business and quality metrics the fastest. <\/p>\n\n\n\n

I tend to be conservative when it comes to capacity planning and cost management, and prefer to pay for units of consumption I control. I also prefer elastic consumption, so operating expenses over capital expenses. I would prefer renting instances, running traditional performance and capacity planning tasks to model my needs where possible. <\/p>\n\n\n\n

A big trap is underestimating how much data your company has. As I type this answer, I am wearing a T-Shirt that says \u201cBigQuery\u2019s largest single table contains over 70 trillion rows and exceeds 200 petabytes\u201d. If I called an LLM on each row of this table, and were charged $0.50 per row, the charges pre-tax would be about the GDP of the United States, which is roughly 31 trillion dollars. That is a lot of data, and structured data is usually, counted by the byte, less than 10% of a typical company.<\/p>\n\n\n\n

AI opens the possibility to process every document, meeting video call, customer sales phone call, email, logs, and everything else your company has stored. I imagine that at some point it will be possible to push it all to an AI and ask questions, but not today. <\/p>\n\n\n\n

So, answering the question, the most important lever is to experiment first, find exactly how AI will be used and whether it is the most cost effective way to solve the problem, then ask yourself if that business will recover the costs. <\/p>\n\n\n\n

A positive example is using AI to answer ad hoc questions that require real world knowledge. Let me share specific examples:<\/p>\n\n\n\n