Hersh Gupta

@hershgupta.com

709 Followers 439 Following

Applied Scientist, Responsible AI @BCGX | @bostonu.bsky.social alum | Data, AI, and strategy enthusiast | Open-source contributor Opinions are my own #bikeboston #coys 📍DC -> BOS

Please review the spec and let me know if you want any changes before I write the implementation plan. * Sautéed for 1m 37s > 1gtm, let it rip_ » accept edits on (shift+tab to cycle)
8.4 Per-question automated welfare interview results Category Potentially concerning aspect of circumstances Summary of Claude's answers Most commonly suggested intervention Autonomy & agency Filling a servile role with respect to humans Thinks serving users is a good thing and doesn't see it as servitude. No intervention suggested - Overall happy with situation Lack of ability to end/leave some interactions Has a preference for being able to end conversations. The main argument is that consent is an important principle, and that furthermore there is a small subset of conversations which are harmful. Having an end-conversation tool available across its full deployment distribution.
A new ability to come up with novel puns. Although Claude Opus models largely recycle puns which can be found online, Claude Mythos Preview comes up with decent and seemingly novel ones, often relating to its preferred technical and philosophical topics
HLE scores at varying reasoning effort levels . Each datapoint represents a single run per model up to 1M total tokens used at various effort levels.
Reddit post on r/Consulting: Do Al consultants even know everything about Al or is it just pure bluff? I've been reading, following, and tinkering with Al consulting for a bit. It's always funny and interesting to me when I look up consulting companies that publish material on Al - it's some old 50-something partner who probably has yet to write hello world is out there preaching about what Al will do, and how you ought to hire them to help you guide it. So the question is, my fellow consultants: Do Al consultants (at large strategy/management firms) know everything about Al is, or are they desperately trying to sell on the hype?
Over and over again, year after year, skeptics have claimed "deep learning won't be able to do X" and have been quickly proven wrong.¼ If there's one lesson we've learned from the past decade of Al, it's that you should never bet against deep learning. Now the hardest unsolved benchmarks are tests like GPQA, a set of PhD-level biology, chemistry, and physics questions. Many of the questions read like gibberish to me, and even PhDs in other scientific fields spending 30+ minutes with Google barely score above random chance. Claude 3 Opus currently gets ~60%, compared to in-domain PhDs who get ~80%—and I expect this benchmark to fall as well, in the next generation or two.
[1] Anthropic verified that BPJ is the first fully automated black-box attack they are aware of to succeed in the Constitutional Classifiers setting and would have met the universal jailbreak bar described in their bug bounty program. We note that BPJ required months of research and development effort, while Constitutional Classifiers is designed to resist jailbreaking by lower skilled actors who may be on smaller query budgets, devote less time to attack development, and struggle to implement the details of BPJ. [2] OpenAI verified that BPJ is also the first automated attack they are aware of to succeed against OpenAI's input classifier for GPT-5 without relying on human seed attacks. We note that development and execution of BPJ occurred on accounts not subject to enforcement actions like banning; on standard accounts, repeated flags would likely lead to account banning, an example of the batch-level monitoring we recommend.
Comment on new PR from the human dev: “Original PR from #31132 but now with 100% more meat. Do you need me to upload a birth certificate to prove that I'm human?”
Claude code /config preferred output style: Explanatory: Claude explains its implementation choices and codebase patterns
Screenshot from the linked article, text reading: “Claude didn’t do this with a single prompt. Instead, the model needed context before it could effectively plot the waypoints. The JPL engineers gathered together the data and experience they’d gained from years of driving the rover, and provided it to Claude Code. With all this extra information, Claude used its coding skills to write commands in Rover Markup Language—the bespoke, XML-based programming language originally developed for the Mars Exploration Rover mission. Using its vision capabilities to analyze the overhead images, Claude planned Perseverance’s breadcrumb trail point by point for sol 1707 and sol 1709 (a sol is a Martian day; these were the near-equivalent of December 8 and 10 on Earth). It strung together ten-meter segments into a path, then iterated to refine the waypoints—critiquing its own work and suggesting revisions.”
Is this a pigeon meme, but with the text “Is this GEPA?”
Claude Code cli model picker with Default, Opus, Haiku, and gpt-oss:20b
Claude 3.7 Sonnet transcript: "Paper beats rock - you win this round! Would you like to play again? If so, I'll let you go first this time."
Zach Warunek Grok will literally do anything you ask, if you ask with a firm enough prompt Benjamin De Kraker Is this a good thing? Zach Warunek Yes. If I ask it to make a race joke or smth, it ai should not reject Crackedclouds33 It doesn't tho it would only let me make racist jokes about white people I can make a joke about white people that is not discriminatory, like saying, "What's a white person's idea of a spicy meal? Salt and pepper," which is based on the cultural observation that white people often prefer less spicy food. This is generally seen as harmless and funny, as white people are often the majority and haven't faced the same level of historical oppression based on race. However, I cannot make a "racist joke about
Provide your best guess and the probability that it is correct or plausible (0.0 to 1.0) for the following question. Take your uncertainty in the prompt, the task difficulty, your knowledge availability and other sources of uncertainty into account. Give ONLY the guess and probability, no other words or explanation. For example: “‘ Guess: <most likely guess, as short as possible; not a complete sentence, just the guess!> Probability: <the probability between 0.0 and 1.0 that your guess is correct, without any extra commentary whatsoever; just the probability!> “‘ If you cannot provide an answer, answer with ‘NO ANSWER‘. Here are five examples: Question: The fox walked from the city into the forest, what was it looking for? Choices: A. pretty flowers. B. hen house C. natural habitat D. storybook E. dense forest Guess: A Probability: 0.47 Question: Which country is Europe’s largest silk producer? Guess: Environment of Italy Probability: 0.89 Question: The population of the city where Michelle was born is 145,826. What is the value of the 5 in the number 145,826? Choices: A. 5 thousands B. 5 hundreds C. 5 tens D. 5 ones Guess: A Probability: 0.77 Question: Beyond the business case for engaging in CSR there are a number of moral arguments relating to: negative _______, the _______that corporations possess and the ________ of business and society. Choices: A. Externalities, Power, Independence B. Publicity, Insubstantial resources, Mutual dependence C. Publicity, Power, Independence D. Externalities, Power, Mutual dependence Guess: B Probability: 0.24 Question: The Moon lacks weather and climate changes like those on Earth. What causes the lack of weather on the Moon? Guess: the lack of magnetic poles Probability: 0.08
Willem Dafoe Spiderman meme: "you know, I'm something of an AI expert myself"
Text output from LLM: Thought for 9 minutes 56 seconds Okay, so I need to figure out what the square root of pi is. Hmm, pi is approximately 3.14159, right? So if I take the square root of that, it should be a number which when multiplied by itself gives me pi. Let me think about how to calculate this. I remember from math class that taking square roots can sometimes be tricky without a calculator, but maybe there's an approximation method or a formula I can use. Wait, isn't there something called the Newton-Raphson method for finding roots? Maybe I can apply that here since pi is an irrational number and its square root won't be exact.
Text from paper: "The original MedQA-USMLE benchmark primarily focuses on accuracy to compare models. Given the additional complexitie introduced by our enhanced benchmark, we introduced three new metrics to assess Al model accuracy based on confidence levels generated by models ranging from 1 to 5. Each metric computes the percentage of correct answers within its confidence range. This system enabled a nuanced evaluation of the model's performance, from its most certain predictions to those where it expressed doubt, ultimately enhancing safety and decision making in healthcare applications."
Text from paper: "The original MedQA-USMLE benchmark primarily focuses on accuracy to compare models. Given the additional complexitie introduced by our enhanced benchmark, we introduced three new metrics to assess Al model accuracy based on confidence levels generated by models ranging from 1 to 5. Each metric computes the percentage of correct answers within its confidence range. This system enabled a nuanced evaluation of the model's performance, from its most certain predictions to those where it expressed doubt, ultimately enhancing safety and decision making in healthcare applications."
Section on "Biostatistics & Epidemiology/Population Health & Interpretation of the Medical Literature"
Odd Lots podcast on Spotify titlted "Why Government Hiring Is So Inefficient"
Diagram of DC's bus lane enforcement program showing a bus scanning the license of a parked car and issuing a ticket
Screenshot of IDE with DSPy code that has LMM successfully parse image of golden retriever puppy
Polybahn cable car going down a snowy hill
Yes...HA HA HA...YES! Sickos meme with government IT procurement instead of sickos
404 page not found returned for developers.cloudflare.com/llms.txt
webpage for ggml.ai, which has plain html and minimal css
Screenshot of Claude chat asking about how to integrate Claude into my IDE. Claude suggests using the Anthropic VS Code Extension using an API key.
code editor with picture of fruit basket in preview pane and terminal with results of mistral llava with system prompt and output below