SuperScript

Back It Off a Quarter Turn

Unknown — Thu, 26 Feb 2026 08:08:20 -0600
A friend actually educated in the workings of LLMs described some aspects of LLM training. A sequence of train-then-verify loops produces models of varying quality. The trainers look for increasing performance, and eventually see it level off or start to decline with overtraining. Then they select the model that exhibited the peak performance, generally not the last.</p>
So the old engineering joke lives on!
How hard should you torque a bolt? Tighten it until it strips, then back it off a quarter turn. </blockquote> </p>
Sorting Networks

Unknown — Wed, 18 Feb 2026 08:41:07 -0600
Today I learned about sorting networks and bitonic sequences. Despite being aware of djbsort</code> since its initial publication, I never looked at any implementation details. Today I ran across this introduction</a> and this apparently canonical reference</a>. Fun stuff!</p>


A Judgement Amplifier
Unknown — Mon, 08 Dec 2025 10:55:57 -0600
At a recent Austin CTO Club</a> meeting someone remarked that AI tool use amplifies the quality of the technology development team members. That is, good developers become better, and bad developers become worse. How can this be so?</p>
AI Tool Effects</h2>
If an AI toolkit is worse at programming than the user, one may expect it to lower the output quality for that developer, degrading their performance. And if the AI tool is better, then similarly one may expect it to raise the quality. It seems natural that training on a wide variety of coding inputs will cause AI to produce average-quality code, not bottom- or top-notch code. In light of this, should we expect regression toward the mean in team member performance?</p>
It is possible that AI is better at programming than all the developers. That seems extremely unlikely. In my experience the tools can readily create a large volume of code, but the quality is mediocre. But supposing that the tools do improve code quality for all developers, how does that explain bad developers becoming worse?</p>
The quality of prompts has a strong effect on the quality of the outputs from the current generation of AI tools. A natural ability to prompt well may correlate with developer quality. Regardless, I would expect training in prompt engineering plus the above regression toward the mean to decrease the differences in developer performance. I suspect there was some such training in the case at hand. This would be an obvious experiment.</p>
Judgement</h2>
My guess is that the quality of judgement</em> in evaluating and applying AI outputs explains the divergent outcome quality. One factor separating good and bad developers is their relative ability to make sound decisions about architecture, code factoring, choice of libraries, testing rigor, code review, and so on. If the AI tools take care of basic blocking and tackling in the development process, then a larger fraction of the remaining human role is applying judgement in making these decisions. And so the quality of these judgements has increased effect.</p>
Can the AI tools decrease the need for such judgement rather than merely displace it? That would require running out of higher-level decisions to pick up after delegating lower-level decisions to the AI tools. I believe we are far from a world in which that condition obtains.</p>
Decision Speed and Quality</h2>
If AI tools do amplify judgement, then decision speed and quality increase in value. Good deciders improve in their roles, and bad deciders become worse. Decisions will tend to have broader scope and effect. Faster and better decisions keep the AI working and pointed in the right direction.</p>
All of this suggests two simple things to try:</p>

Test for role-related decision speed and quality in your hiring process.</li>
Work to improve decision speed and quality in your employees.</li>
</ul>
If successful, these will put people on the plus side of the ledger as AI tool users.</p>


Making the Cut
Unknown — Thu, 04 Dec 2025 07:40:52 -0600
“I put the ticket in more than two years ago and it still isn’t done. When will you get to it?” Our Product lead apparently looked at our backlog as a queue, in which everything that goes in is implemented sooner or later. That makes sense if your tickets are for customer support. But not for a product backlog.</p>
I replied that if her request had failed to achieve priority in two years it was never going to get priority. Other things arrived, maybe before and certainly after, that jumped ahead in line.</p>
Following that encounter I announced to everyone that in two weeks I would purge all tickets older than 18 months. If they wanted to keep an item in the backlog, all they needed to do was to update the ticket. They could simply add a “Keep this” comment.</p>
When the time for the purge came, about a dozen old tickets among a couple thousand had been updated. The rest were purged. And after that I repeated the exercise, bringing the threshold down to 12 months. Nobody missed the purged items.</p>
Backlog, Not Queue</h2>
The idea that a product backlog is a queue in which every input is acted on overlooks the main value of a backlog: things drop off the bottom. A backlog delivers value by determining what work will not</em> be done, as well as by prioritizing the work that makes the cut.</p>
One fundamental problem shared by virtually all electronic product backlog tools is that they remember too much. No doubt you have had that conversation in which someone says “Archive it in case we need to reinstate it later.” or “Let’s save it so that we can look back at how we reached that decision.” It sounds good. But the standard critique of waterfall planning applies here. When you reinstate or look back at an item you have a different, more fully informed context. Your earlier considerations lacks the benefit of the learning you did since setting aside the item. Picking up that old work again requires mapping it into your current context, and working your way through those same considerations in light of your current knowledge. Reviving your reasoning or the ticket itself is not so straightforward as it first appears. A good rule of thumb is “If it is important enough, it will come up again.”</p>
Items drop off a backlog not because they embody bad ideas, but because they are not good enough. An individual item cannot be measured in stand-alone fashion on its own merits. Instead it must make the cut in competition with the other items in the backlog. The decisions about what to prioritize and where to draw the line are not easy. Nevertheless, they are key to achieving focus on the items that will be developed.</p>
Trade-Offs</h2>
When your backlog is large, you spend a lot of time and effort on reviewing, updating, and reprioritizing it to reflect new learning. And you will invariably change or set aside some of the items in the backlog after this investment of time and effort.</p>
Those changes and set-asides are not necessarily dysfunctional. Completing every item in your backlog without changes suggests that you are not learning as you go, because learning modifies and improves the way you approach work. And learning that an item is not worth doing frees you to do more important work. So changes and set-asides can be valuable. They can also be taken too far. A high rate of set-asides suggests that your backlog is either too large or too ill-defined. The sweet spot sits in the middle.</p>
How big and how well-defined should your backlog be? This is a question of trade-offs. Do you spend inordinate amounts of time on review, updates, and prioritization? That suggests that your backlog is too large. Do you regularly radically change backlog items? That suggest that your backlog is ill-defined. Do you set aside a high proportion of your backlog items? That suggests it is too large and also ill-defined.</p>
One rule of thumb is to size the backlog by the look-ahead it provides. Make it large enough to include work you expect to complete in some reasonable amount of time plus work coming up in the next such interval.</p>
Conclusion</h2>
The backlog is your tool for managing these trade-offs. Its main benefit is giving you a place to make the cut, and set aside the things that don’t belong in view now. Make the cut, give yourself focus, and don’t fall prey to the temptation to remember everything and your flow will improve.</p>


Prototype vs Implementation
Unknown — Sun, 30 Nov 2025 13:27:30 -0600
The Austin CTO Club</a> recently met to discuss Managing C-Suite Expectations</a>. One specific point received little attention that evening, and in my opinion deserves additional thought: prototype versus implementation.</p>
When presented with a prototype, executives frequently think of it as an early implementation. Many of the concerns that separate prototype from implementation are invisible to executives, and reasonably so: security, scalability, deployment, integrations, and so on.</p>
Without knowing about how easy or difficult these concerns are to address, it’s no wonder that time to production deployment is their focus. From where they sit it seems that product development stalls after the creation of the prototype. And in the meantime they have other concerns, including evaluation of additional prototypes. This dynamic leads to tension during implementation between the need for careful attention to security etc. and the need to move quickly to market.</p>
Vibe coding is on the rise. It shortens the time to prototype creation as well as opening the door to prototype creation to those with less technical knowledge. An increasing number of product managers can use AI to produce prototypes. We will see the number of prototypes produced increase. My guess is that it will rise by a couple orders of magnitude. And then what happens?</p>
Mind the Gap</h2>
How does everyone react to this superabundance of prototypes? And what does this mean about the gap between prototype and implementation? How do we help executives operate in this prototype-rich environment? I’ve heard at least four different ideas for addressing this.</p>
Ignore It</h3>
Ignore the problem, and hope it goes away. Doing nothing is usually an option, and often an underrated one. In this case, ignoring it seems likely to exacerbate the problem unless and until the AI tools that speed up prototyping become good enough to address the additional concerns of implemention. In other words, if the tools become sufficiently capable, the gap between prototype and implementation closes, eliminating the problem.</p>
If you bet on the gap disappearing and it does not, where does that leave you? And what can you do in the interim when the gap does exist? What if the gap closes, but then your production implementation exhibits some bug? Who fixes that if those who implemented it lack deep technical know-how? This approach seems very high risk, though perhaps it helps a smaller company get to market more quickly.</p>
Dumb Down</h3>
Some people advocate for creating prototypes that look more like wireframes than polished applications, giving a visual clue as to the incompleteness of the implementation. This would highlight that something is a prototype and not an implementation.</p>
The entire reason for polishing an application is to make it more attractive. Sticking to cruder presentations foregoes that benefit. Can the less attractive option win over the more polished alternative? That seems doubtful, especially if you believe better UX and design add value for the user.</p>
Flood the Zone</h3>
What if you create enough prototypes to overwhelm those who evaluate them? The superabundance of something reduces the value of each one, so each one is thereby easier to dismiss. The low cost of producing any one prototype does lower the opportunity cost of your selection of which one to implement.</p>
Does the superabundance of prototypes lead to greater ambition, and attempts to select more options to implement? This sounds like a new expression of the original problem. Expectations scale much more readily and quickly than capacity does. If prototype capacity doubles, how much executive ambition for implementations scale?</p>
Experimentation</h3>
Treating prototypes as experiments that explore implementation options, and the choice of implementation as outcome of those experiments, draws a business-related distinction between prototype and implementation. This is easier to convey to non-implementors than direct implementation concerns. It also leverages the role of non-implementors in increasingly important decision making.</p>
This approach does require tempering the scaling executive ambitions, just as with flooding the zone. In this case is the scaling on the prototype side gives wider choice among different implementations, letting non-implementors improve the quality of outcomes through their choices, even if implementation capacity does not rise.</p>
Other Approaches?</h2>
There are doubtless many more ways to address the question with some combination of constraint, education, and planning. What approach would you take?</p>


Document-Driven Development
Unknown — Mon, 24 Nov 2025 18:20:57 -0600
In recent months I’ve spoken to many people about inverting the model and coding my docs rather than documenting my code. Today I learned that this has been developed under the heading of Documentation-Driven Development</a> or Docs-Driven Development</a> or Document-Driven Development</a> or Readme Driven Development</a>. I found it via the third link</a>.</p>


Defining Hypotheses
Unknown — Thu, 20 Nov 2025 11:03:11 -0600
Startup founders must engage in a lot of experimental learning to find and confirm their markets. The vision matters, of course, but it also must survive its test in the market. To that end, formulating good experiments is essential.
The heart of an experiment is the hypothesis it tests. What do you expect to happen? How do you measure the outcome? What is your threshold for success? A poorly defined hypothesis clouds the outcome and impedes the search for clear signal. So here are two pieces of advice for startup founders on how to formulate a good hypothesis.</p>
First, make it boolean. Make the hypothesis yield a pass/fail result. This is the simplest possible set of outcomes. Second, include your criteria for success or failure in the hypothesis.</p>
These two points are important for the same reason. If you apply each of them and then run your experiment, you will have a clear outcome: we passed or failed according to criteria established at the outset. With that clarity, you will discuss what it means that you passed or failed. This is the forward-looking question you need to address. It focuses on how you will behave in light of the result. What changes you can make your hypothesis? How does the result alter your vision? What strategy you should adopt in response to this new finding? And so on.</p>
In contrast, if you do not establish criteria beforehand, or if you produce a complicated set of outcomes, then you will discuss whether or not you passed or failed. That is a backward looking question, one typically loaded with emotion from wanting to succeed by passing. It does not focus on options for reacting to the outcome, and instead obsesses over the outcome itself.</p>
This is simple and yet in practice proves difficult. I’ve seen more hypotheses that violate these two rules than ones that adhere to them. Defering the formulation of clear success criteria hedges your bet. You can adjust them later so that your experiment succeeded by passing. That generally makes people feel good because they can claim to have been right all along. But it involves a mistaken notion of experimental success.</p>
The success of an experiment hinges on the learning that it provides. Both pass and fail outcomes support learning. There is no reason to believe that you learn more from “pass” than from “fail” when running an experiment. If all of your experiments succeed, you are playing it too safe and not stretching far enough into the unknown. If all of your experiments fail, then you are out of touch with your own capabilities. A good rule of thumb is to aim for a 50% success rate. If you read “pass” as “success” then that sounds abysmal. But both “pass” and “fail” contribute to experimental success.</p>
Finding the right thresholds to apply in your pass/fail criteria is not always easy. You are dealing with unknowns when learning, so of course you will miss the mark with some frequency. With success criteria defined in advance you can always decide that the failure was due to erroneous criteria and formulate a new hypothesis. But if you establish the criteria after the fact you will have lost the clear signal from your experiment. That may feel better, but it does less to advance your understanding of the market, and diminishes the objectivity of the outcome. Market understanding matters far more than your feelings.</p>
Over time as a product matures, the experiments will naturally trend toward changes of lesser consequence. You don’t want to make a major change to a deployed system that fails your user 50% of the time. But that’s not the same question as pass/fail of your experiment. Instead of changing the pass rate in your experiments, reduce the scope of impact. Entrepreneurs usually have experience working in larger companies. And they often bring with them the practice of making narrow-scope changes and conflating pass with success. In an immature business you do not have time for that. Scope your experiment appropriate to the risk of failing to learn.</p>
In summary, restrict yourself to pass/fail results, and agree on your success criteria before running your experiment. Doing so you will give you the clarity needed to focus forward and adapt.</p>


MVPs Reveal Preference
Unknown — Fri, 03 Oct 2025 14:40:37 -0500
Paul Samuelson</a> introduced the concept of “revealed preference” in a 1938 article entitled A Note on the Pure Theory of Consumers’ Behavior</em>. It means the preference implied by a consumer’s behavior in the market. It’s the economist’s broadening of putting your money where your mouth is.</p>
What does this have to do with MVPs? An MVP is a tool to reveal preference.</p>
I once worked with a team that wanted to build a mobile app to help people rate their phone carriers. The team started with a paper prototype MVP. They showed it to people and asked them what they would say about their carrier. People responded with great enthusiasm and told them all sorts of things positive and negative about the carriers.</p>
But team learned only “stated preference”, the preferences expressed in response to questions or a survey. They didn’t learn revealed preference, how people behave when given the choice.</p>
So I sent them out again. This time they introduced the prototype and simply handed it to people with a pencil and asked them to fill it out. The response was completely different. Almost everyone declined to participate. The revealed preference was to avoid the app, not to engage with it.</p>
When building a product, how do you learn how users will interact with it? Will they in fact adopt it? How will they behave when they use it? Will they prefer it to the alternatives? How much is it worth to them? These are all questions of revealed preference.</p>
A successful product engages users, changes their behavior, and improve their lives. Users may say one thing and do another. What matters is their behavior, their revealed preference. And the proper use of an MVP is to reveal that preference. If you genuinely understand users’ revealed preference, you can build a product that serves them well.</p>
Jeff Patton</a> has a good video</a> on the history and interpretation of the term Miminum Viable Product, and says “Product” really means “Test”, an experiment to test some hypothesis. More recently Gagan Biyani took up the same refrain</a>. Regardless of what you call it, focus your hypothesis on revealed rather than stated preference, and build an MVP to test it.</p>
An MVP is a tool to reveal preference.</p>


Cheap and Cheerful
Unknown — Sat, 20 Sep 2025 13:41:17 -0500
If you shop at Costco you are likely aware that they give away boxes of many shapes and sizes. Many of these are reinforced to protect delicate contents such as ripe fruits. And most have at least one low-cut side. These qualities make them excellent impromptu shelf organizers, say for a metro rack.</p>




</figure>

</div>
The author of 2-Second Lean</a> has a YouTube channel</a> that includes virtual tours of lean organizations. The best one I’ve seen is the (second) tour of Premier Group</a>. The video pertains to manufacturing, but even for those of us in software development it is chock full of ideas. A strong emphasis on culture and growth of people runs through it start to finish. It’s well worth a look.</p>
Owner Bernard Powell uses two noteworthy phrases that I want to map into the world of software development:</p>

Cheap and Cheerful</strong>: Improvements that don’t cost a lot. For example, recycling discard material into improvements. This is relatively easy with respect to your desk or office area, or a common work room. What about in software? Small meta-programming improvements may qualify. Simple improvements in your daily kit, like improved editor keybindings may qualify. Simple automation has to be the big example. If you have done it for a third time, ask yourself whether you can automate it?</li>
Walk-Away Standards</strong>: How should something look when you are done with your work? That’s one thing in a physical environment like a factory floor. But how does it work in software? Standards around documentation, code review, testing, code formatting, etc. may be the right analogy here. How can you leave things so that you or a colleague can pick up right where you left off without difficulty, and without a lot of work on your part.</li>
</ul>
He also emphasizes the utility of scoreboards by analogy with sports: If the stadium had no scoreboard, nobody would come to watch the game. So keep track of the winning and losing.</p>
What prevents us from applying these deas to greater effect in software development?</p>
Some developers want to focus on coding and don’t want to engage fixing process. Some object to the rate of change resulting from continuous improvement. Some think that their processes are too complex and variable for focused improvement. I don’t believe it.</p>
It’s relatively easy for developers to understand that software developed but not deployed is inventory, one of the standard 8 wastes</a>.</p>
I want to map this completely into software development, in cheap and cheerful fashion.</p>


Letters in LaTeX
Unknown — Mon, 08 Sep 2025 16:38:05 -0500
Once upon a time I did a great deal of work with  \(\TeX{}\) and \(\LaTeX{}\). I implemented many elements that were not broadly supported at the time: changebars, backdrop images, special page construction processing, letter spacing, color, and so on. After finding a but I wrote up a proposed fix and mailed a letter to Donald Knuth</a>, who responded, returning my letter with notes indicating that there was a better way to fix the problem.</p>
Shortly thereafter he called me to ask what led me to the discovery. It was letterspacing, “a dirty word around here” he said.  My work moved into a different realm, and so I didn’t keep up, but as I recall it the next release of \(\TeX\) had numerous bugs. I don’t know whether the fix to mine contributed to the new crop.</p>
I receive a check for my efforts, and another one later for finding a typo in the \(\TeX\hbox{Book}\). Sadly they long ago faded into an unreadable state.</p>
But today I needed only to create a \(\LaTeX{}\) style file for letterhead. It started very slowly, but then suddenly it seemed familiar again. And now I have an easy and even scriptable way to produce decently-formatted letters. The tools are venerable, not obsolete.</p>


Google Sheet Sort
Unknown — Fri, 05 Sep 2025 09:23:27 -0500
Today I learned about the sort</code> function in Google Sheets. This came in handy to show different sorted views of one sheet on another. Combining a drop-down with a nested IF</code> statement allows selection of the sort to apply on the data in one sheet to populate another sheet.</p>
The sheet Contacts</code> contains the primary data. A Sorted</code> sheet contains a drop-down in A1</code> with values Latest</code> and Priority</code>. The sort appears in B2</code>, under the headers:</p>
=if(A1="Latest",sort(Contacts!A2:BI,2,false),if(A1="Priority",sort(Contacts!A2:BI,1,false,4,true,6,true)))</span></span></code></pre>
And in Sorted!B2</code> this formula copies over all of the headers from Contacts</code>:</p>
=arrayformula(Contacts!A1:BI1)</span></span></code></pre>
No more manual sorting!</p>


Hot-Potato Package Delivery
Unknown — Wed, 03 Sep 2025 07:51:03 -0500
I ordered something from Walmart. They shipped it to me. Their delivery tracking said that USPS handled the final delivery. And USPS says that on date X they “Delivered, Front Door/Porch”. They did not. Could it be in the mailbox? Nope.</p>
Two days later I received a mystery package from Amazon. It was my Walmart order! Yet Amazon appears nowhere in the tracking data.</p>
So, to handle the last mile delivery Walmart tossed the hot-potato package to USPS, and USPS to Amazon, and finally Amazon to me.</p>
I’m thankful that the package arrived, because chasing down a lost package through this delivery chain would probably be a hopeless endeavor, and certainly not worth the time.</p>


Idempotent Home Directory Setup
Unknown — Sat, 30 Aug 2025 17:55:41 -0500
When I set up a new FreeBSD jail, I pull my home directory configuration from a GitHub repo. When I initially put this together I didn’t ensure that it was idempotent. Now it is, with the addition of a test and alternation for each operation that is not itself idempotent.</p>
#</span>_# setup.home: Configure the home directory repo for updates</span></span>
setup.home:</span></span>
	@git</span> remote</span> get-url</span> origin</span> 2</span>></span>/dev/null</span> |</span>|</span> git</span> remote</span> add</span> origin</span> git@github.com:superscript/home.git</span></span>
	@git</span> show-ref</span> -</span>-quiet</span> -</span>-verify</span> refs/heads/host</span> |</span>|</span> git</span> branch</span> -</span>m</span> master</span> host</span></span>
	@git</span> fetch</span> origin</span></span>
	@git</span> show-ref</span> -</span>-quiet</span> -</span>-verify</span> refs/heads/master</span> |</span>|</span> git</span> co</span> -</span>b</span> master</span> origin/master</span></span>
	@git</span> merge</span> -</span>-ff</span></span></code></pre>
This allows me to use setup.home</code> as a dependency for the broader setup targets.</p>


The Dismal Science
Unknown — Fri, 29 Aug 2025 07:56:26 -0500
Today I learned that the “Dismal Science” description of economics comes from a Thomas Carlyle essay</a>. He argues there against economic reasoning ‘which finds the secret of this Universe in “supply and demand,” and reduces the duty of human governors to that of letting men alone’, and specifically because it supports the ‘sacred cause of Black Emancipation’.</p>
Count me as dismal, then.</p>


A Two Card Strategy
Unknown — Tue, 26 Aug 2025 20:49:26 -0500
Today I discovered that I was the victim of an instance of credit-card fraud. I disputed the charge. The purchase was labeled as “card present” at a store I’ve never been to. My card was never present at that store.</p>
The same thing happened to me once before with a card issued by Citi. When they concluded their investigation they falsely claimed that the charge was legitimate, and demanded that I pay it. As in today’s case, the charge was made in a store that I and my card never visited, and labeled “card present”. Their faulty conclusion cost me hundreds of dollars. I no longer give Citi my business. They appropriated enough of my money through their false claims to deserve my business.</p>
Today’s instance of fraud looks similar. I will not be surprised if another false claim of legitimacy comes out of the “investigation” into the charge. And if it does, a second bank will lose my business.</p>
But enough about banks confiscating money from their customers based on false claims of charge legitimacy! I did learn an important lesson from today’s events. This card had a mix of subscription and ad-hoc charges on it. I should have been following a two-card strategy to separate the two.</p>
Ad-hoc charges require exposing the card to unknown point-of-sale equipment. That’s a much higher risk payment than one arranged over the wire directly with the vendor and then charged directly to the card electronically, all most likely run through a tokenizing processing vendor. Separating the ad-hoc from recurring purchases divids the high from the low risk charges.</p>
The ad-hoc charges are more likely to be compromised. But they are not difficult to change. You simply use a different card. The recurring charges, in contrast, are extremely painful to change. You need to update payment methods, make sure there is continuity of payments, make sure that no scheduled payments hit the old card if it has been disabled, etc. And that’s what I had to do today.</p>
So from now on I will follow a two-card strategy, and split the two types of charges. I’m surprised I never heard this suggested before. It was a painful lesson, but useful.</p>


Interviewing With AI
Unknown — Tue, 19 Aug 2025 06:33:32 -0500
The vibe-coding trend is alarming to anyone who values correctness in code. Putting AI generated code into production without full scrutiny adds operational risk to the extent that your system relies on correctness. With increasing use of AI tooling in development, how do you ensure that your new hires can both take advantage of the tools and also ensure correct execution?</p>
At a recent meeting of the Austin CTO Club</a> a member shared with me a useful approach to this problem when interviewing developers in an AI-enabled context. The idea is to split the interview into two parts that focus on each element in the question, first how to use the AI tools, and second understanding the outputs from those tools.</p>
It works like this:</p>

Give the candidate a simple project to implement with the help of AI.</li>
Timebox the implementation.</li>
When time runs out, have them demo the results to you.</li>
Ask them to explain how the system works.</li>
</ul>
Candidates who are good tool users will do well on the implementation. Candidates who are highly technical will demonstrate good understanding of the system in their explanation. You want those who do well on both implementation and explanation.</p>
As with most hiring decisions, you must gauge the upside potential of the candidate in your context. If your business is extremely sensitive to errors (say, fintech or medical) you may place higher weight on the understanding. If your business is less sensitive to errors (say, social media) then you may weight the tool use more heavily. In either case, this technique gives you good inputs to weigh.</p>
The AI tools for development are changing rapidly. How frequently do they and will they produce correct outputs? Can a candidate keep up with this fast tool evolution? Does your environment support continuous improvements in tool use? Factoring your answers into your weights amounts to longer-term bets on tool capabilities, candidate, and your organization. Good judgement in hiring is always difficult. This technique provides important inputs for those judgements.</p>


Scripting asciinema
Unknown — Mon, 18 Aug 2025 13:46:46 -0500
The asciinema</code> program captures shell interactions in a form that can be replayed, or turned into an animated gif file. It’s an extremely convenient way to demonstrate command-line operations and utilities. But used directly it has one basic weakness: the need to type in the commands.</p>
After starting to use asciinema</code> this week, the annoyance of typing in the commands became immediately unbearable. My solution is called asciinema-script</code></a>. This program records an interaction that is itself scripted. For example, here is a simple script:</p>
t</span> bin/asciinema-script</span></span>
s</span> 1</span></span>
t</span> bin/asciinema-script</span> help</span></span></code></pre>
Using asciinema-script</code> to produce a gif from this script yields the following:



</a>
asciinema</figcaption>
</figure>
</p>
Another post</a> uses this script:</p>
t</span> bin/pipeline</span></span>
s</span> 1</span></span>
t</span> bin/scrip</span> help</span> bin/pipeline</span></span>
s</span> 1</span></span>
t</span> bin/pipeline</span> ::</span> echo</span> a</span> ::</span> sh</span> -</span>c</span> '</span>echo b && cat -</span>'</span></span>
s</span> 1</span></span>
t</span> bin/pipeline</span> ::</span> echo</span> a</span> ::</span> xargs</span> echo</span> b</span> ::</span> xargs</span> echo</span> c</span></span></code></pre>
It’s now almost trivial for me to create a reproducible demo.</p>


Pipelines Without Pipes
Unknown — Sun, 17 Aug 2025 16:31:27 -0500
The pipeline</code> program constructs a pipeline from its arguments. The first argument is the separator. The program splits the command line on separator arguments and constructs a pipeline from the components.</p>
Here’s what it looks like in use:



</a>
pipeline</figcaption>
</figure>
</p>
What good does this do? If you dynamically assemble a pipeline in a shell program, you must pass it through shell parsing in order to execute it. This brings all of the difficulties of argument quoting into play. The pipeline</code> construction does this in a safe way. The caller can manipulate the entire pipeline as a set of ordinary arguments without worrying about interpolation problems.</p>
The underlying code looks like this:</p>
pipeline_cmd</span>(</span>)</span> {</span></span>
  local</span> prefix</span>=</span>"</span>$</span>1</span>"</span></span>
  local</span> sep</span>=</span>"</span>$</span>2</span>"</span></span>
  shift</span> 2</span></span>
</span>
  local</span> cmd</span>=</span>'</span>'</span></span>
  local</span> i</span>=</span>3</span></span>
  local</span> p</span>=</span>'</span>${1}</span>'</span></span>
  for</span> a</span> in</span> "</span>$</span>@</span>"</span></span>
  do</span></span>
    if</span> test</span> "</span>$</span>a</span>"</span> =</span> "</span>$</span>{</span>sep</span>}</span>"</span></span>
    then</span></span>
      cmd</span>=</span>"</span>$</span>{</span>cmd</span>}</span> |</span>"</span></span>
      p</span>=</span>'</span>${1}</span>'</span></span>
    else</span></span>
      cmd</span>=</span>"</span>$</span>{</span>cmd</span>}</span> \"</span>$</span>{</span>p</span>}</span>\$</span>{</span>$</span>i</span>}</span>\"</span>"</span></span>
      p</span>=</span>'</span>'</span></span>
    fi</span></span>
    i</span>=</span>$(</span>(</span>i</span> +</span> 1</span>)</span>)</span></span>
  done</span></span>
</span>
  printf</span> '</span>%s\n</span>'</span> "</span>$</span>{</span>cmd</span>}</span>"</span></span>
}</span></span>
</span>
#</span> pipeline prefix sep prog1 [sep prog2 ...]</span></span>
pipeline</span>(</span>)</span> {</span></span>
  eval</span> "</span>$(</span>pipeline_cmd</span> "</span>$</span>@</span>"</span>)</span>"</span></span>
}</span></span></code></pre>
The pipeline</code> program calls the pipeline</code> function with an empty first argument. You can find the code here</a>.</p>


Friedman, Pigou, Coase
Unknown — Sat, 16 Aug 2025 15:26:57 -0500
In a rightly famous 1960 meetingFamous among economists, at least. Sadly, no attendee recorded the meeting.</span>
 Ronald Coase</a> persuaded a room full of University of Chicago economists that his view on what we now call externalities</a> and transaction costs (the costs associated with market exchange) was correct. His argument subsequently appeared in the very readable The Problem of Social Cost</em></a>. In a delightful interview</a> with Richard Epstein, Coase likened his claims to asserting \(2+2=4\).</p>
I was once privileged to speak at some length with Deirdre McCloskey</a>, who repeated a statement from The So-Called Coase Theorem</a> that ‘something like a dozen people in the world understand that the “Coase” theorem is not the Coase theorem’. I’m presumably outside of that group. Nevertheless, I will try to provide some intuition from one non-economist to another.</p>
The primary insight in The Problem of Social Cost</em> is typically Coasean: beautiful, simple, and deep. It is that an externality is a joint production between two parties: a polluter and a nearby resident; a noise maker and someone within earshot; a spark-producing train and a grain farmer (Coase’s example).</p>
When my neighbor blasts his music loud and I’m not home, there is no conflict. When I’m home and he is not blasting music, there is no conflict. But when he is blasting and I am home, together we produce a conflict. The conflict is transactional</em>. You cannot speak meaningfully of one side imposing harm on the other, because the conflict is a joint production.</p>
Normal economic reasoning about transactions applies. The terms of the interaction are potentially subject to negotiation. What makes it worthwhile for each party to accommodate the other? Can they agree on terms? Compensation of one kind or another could change hands to satisfy an aggrieved party. Is it feasible to reach such an arrangement?</p>
Transaction costs associated with this negotiation are an obstacle to its resolution. Therefore one should look to reduce transaction costs to improve the chances for the two parties to resolve the conflict.</p>
That’s my understanding of Coase’s meaning in The Problem of Social Cost</em>.</p>
The pre-Coasean view was based on work by Arthur Pigou</a>. In this view, one party imposed costs or conferred benefits on another in the case of externalities. For imposed costs, a tax should apply to the offending activity to cause the offender to internalize the costs of the externality on the offended party. One party produces the conflict, and the other is its mere victim. There is no notion of transactional conflict.</p>
The contrast between the pre- and post-Coasean rhetoric is stark. In the pre-Coasean case one speaks of “imposed costs” and interventions in order to force the imposer to “internalize” them. In the post-Coasean case the question is one of transaction costs limiting options for negotiated settlement.</p>
Transaction costs cannot always be reduced sufficiently to resolve a conflict. A standard example is air pollution. In some cases you have many people on one side of the externality transaction (a factory producing smoke in a neighborhood) or many on each side (drivers emitting smog, and bystanders breathing it). Transactions costs could be so high as to prevent any hope of negotiated resolution. Maybe a Pigouvian tax is the best one can do. But even then, we should not abandon the post-Coasean view. The externalities remain transactional, even if impossible to resolve by negotiation.</p>
Watching Peter Robinson</a> interview</a> Milton Friedman</a> prompted me to write this blog. When they discuss the pollution, Friedman explains the desirability and difficulty of negotiated resolution of the conflict, but several times he mentions one party “imposing a cost” on the other. In his defense, he sometimes qualified with “uncompensated”, but not always.</p>
I have little doubt that Friedman understood Coase well. He was reportedly</a> the leading questioner in that famous meeting. But here he wandered between the pre- and post-Coasean rhetoric, even if not between the views. Every utterance of “imposing a cost” sounded like a small step away from Coase’s fundamental insight about the joint production of externalities.</p>


How Fast Is Gas?
Unknown — Sun, 10 Aug 2025 19:01:04 -0500
I gave this prompt to Claude today:</p>
I want to estimate the total gasoline fueling capacity in the United States. The key questions are:</span></span>
* How many operational gas pumps are there in the United States?</span></span>
* What is the speed at which an individual pump can fuel a vehicle?</span></span>
* How much non-fueling time contributes to pump occupancy, and therefore fueling speed in realistic use?</span></span>
* What is the average mileage of a vehicle?</span></span>
* What is the aggregate speed of all of the capacity in MPH?</span></span>
Research these questions, come up with estimates, cite your sources of information, and indicate the numbers that come out of your investigation. Then calculate the total capacity in MPH.</span></span></code></pre>
The bottom line: This conservative estimate of practical gasoline fueling capacity in the United States is between 4.2 and 5.0 times the speed of light!</p>
Here is Claude’s response in full.</p>
US Gasoline Fueling Capacity Analysis</h1>
The United States gasoline infrastructure can theoretically support approximately 3.11 billion miles of driving per hour</strong>, representing massive untapped capacity that operates at only 24% utilization due to operational inefficiencies rather than physical constraints.</p>
Current infrastructure capacity</h2>
Pump Infrastructure Scale</strong>
Based on industry data from NACS and the American Petroleum Institute, the US operates 150,000-152,000 gas stations</strong> with an estimated 900,000 individual fuel dispensers</strong>. The convenience store segment dominates, with 121,852 locations selling fuel and accounting for 80% of gasoline sales nationwide.</p>
Technical Performance Specifications</strong>

Modern retail gas pumps operate at 8-10 gallons per minute (GPM)</strong>, with the EPA regulatory maximum set at 10 GPM for passenger vehicles</strong>. This limit stems from vapor recovery system constraints and safety considerations rather than mechanical limitations. Commercial truck dispensers can achieve 20-40+ GPM, but the vast majority of retail infrastructure serves passenger vehicles at the 10 GPM standard.</p>
Real-world operational efficiency</h2>
Massive Underutilization Discovered</strong>
Analysis of 30,000 gas stations reveals average pump utilization of only 24%</strong>, with high-performing stations reaching just 41% during peak hours. This means pumps sit unused 59-76% of the time, creating an enormous capacity bottleneck unrelated to technical capabilities.</p>
Transaction Time Breakdown</strong>
While theoretical fuel dispensing for an average 11.7-gallon fill-up should take 1.2-1.5 minutes at optimal flow rates, total customer visits average 5-8 minutes</strong>. The effective “duty cycle” shows pumps spend less than 40% of transaction time actually dispensing fuel, with the remainder consumed by payment processing, customer positioning, and selection delays.</p>
Vehicle fleet considerations</h2>
Current Fleet Efficiency</strong>
New vehicles sold in 2023 achieved 27.1 MPG real-world fuel economy</strong>, but the actual on-road fleet average is significantly lower due to older vehicles still in service. Conservative estimates place the true fleet average around 24 MPG</strong> when accounting for the full range of vehicles currently operating.</p>
Efficiency Trajectory</strong>
Regulatory pressure drives toward 49 MPG for new vehicles by 2026, while electric vehicle adoption reached 11.5% of production in 2023. However, continued consumer preference for SUVs and trucks (75% of new sales) partially offsets technology improvements.</p>
Total capacity calculation</h2>
Methodology and Results</strong>
Starting with 900,000 dispensers operating at 10 GPM maximum capacity yields 9 million GPM theoretical capacity. Applying the observed 24% utilization rate reduces this to 2.16 million GPM effective capacity</strong>, equivalent to 129.6 million gallons per hour</strong>.</p>
Converting to driving capacity using the estimated 24 MPG fleet average produces the final result: 3.11 billion miles per hour</strong>, with reasonable ranges from 2.85 to 3.37 billion MPH depending on fleet efficiency assumptions.</p>
Capacity utilization insights</h2>
Scale and Opportunity</strong>
This infrastructure capacity could theoretically support 74.6 billion miles per day</strong>, while actual US vehicle miles traveled totals approximately 8.8 billion daily. Current operations represent only 11.8% of total infrastructure capacity</strong>, indicating enormous unused potential.</p>
Primary Constraints</strong>
The limiting factors are temporal demand clustering, geographic distribution mismatches, customer behavior patterns, and payment processing inefficiencies. During peak fueling hours (morning and evening rush periods), utilization increases to 41%, but 59% of capacity remains unused even at maximum demand.</p>
Conclusion</h2>
The United States possesses gasoline fueling infrastructure with extraordinary theoretical capacity - enough to support nearly three times the current national driving demand every hour. The constraint is not physical infrastructure or technical capability, but rather operational efficiency and demand timing patterns. This analysis reveals that expanding gasoline capacity through additional stations may be less critical than optimizing existing infrastructure utilization through improved payment systems, customer flow management, and operational efficiency improvements.</p>


The More You Owe the More You're Worth
Unknown — Fri, 08 Aug 2025 15:51:09 -0500
Today I read an old Econlog post</a> by David Henderson. He links to this gem</a>.</p>


2 Second Lean
Unknown — Wed, 30 Jul 2025 13:52:01 -0500
Two weeks ago at Lean Coffee Austin two gentlemen from a construction company showed up. It turned out we all share an affinity for 2-Second Lean</a> by Paul Akers. I originally discovered the book almost 15 years ago, not long after it was published, but had not looked at it for a number of years. A fresh look confirmed my earlier opinion of the book as the best combination of fun application and solid principles I’ve seen in the Lean Management literature.</p>
The book chronicles Akers’s journey into Lean Management at his company FastCap</a>, from pride in his excess inventory through obsession with waste elimination, to creating a company culture that challenges everyone to improve continuously.</p>
The book includes stories about Akers traveling to Japan to study companies that implemented Lean Management cultures. The companies are indeed exemplary. But the lesson from these visits is not to imitate what he saw there. Adopting the practices without understanding is a cargo-cult approach. Adopt the principles, and develop the practices to suit your needs and circumstances. Make the principles your own, and the rest will follow.</p>
The primary innovation of the book rests on a simple observation: Big kaizen events are less in keeping with the principle of continuous improvement than are frequent small improvements. While straightforward and clear, this observation has deep implications.</p>
The opportunities for small improvements are everywhere. Akers urges the reader to ask “What bugs me?” and then fix it. And then start again in an endless cycle.</p>
Akers identifies three pillars of his approach to Lean culture:</p>

Learn to see waste.</li>
Improve every day.</li>
Make before and after videos of your improvements.</li>
</ul>
He sets the expectation that everyone in his organization will do these things, every day.</p>
Why does 2-Second Lean work? I see three basic reasons.</p>
First, the opportunity for small improvement is everywhere. How many things bug you during the day? Every one is another opportunity. Focusing on what bugs you makes it easy and provides some degree of immediate gratification.</p>
Second, even small improvements add up over time. Daily 2-second improvements accumulate to significant effect, especially when they involve frequent operations. And some of will be large, accruing benefit much more quickly.</p>
Third, and most importantly, the constant search for 2-second improvements puts you in the mindset of finding and eliminating waste. That equips you to find larger opportunities for improvement.</p>
What bugs you? Unless you are hopelessly complacent, you have a long list. Fix something. Then repeat.</p>


My Best Losing Bet
Unknown — Wed, 18 Jun 2025 09:05:39 -0500
Nothing focused him more than a bet. A friend who interviewed with him shortly before Halloween had made a giant squid costume. “Dude, you’re going to bring your costume when you visit and wear it in the office!” he said, and then went around betting everyone that he could make a candidate wear his Halloween costume in the office. Of course he won.</p>
One night at the local dive bar he approached me wild-eyed and asked out of the blue “How much would you bet that I can’t do 30 chin-ups? A dollar? Five dollars?” I looked him up and down. Short. Broad shouldered. Big chested. Solid. About 50 pounds overweight. The frame of a gymnast, hopelessly out of shape and dumpy. I answered “A hundred dollars.”</p>
He looked momentarily stunned. Whatever his expections, this fell outside of them. Then bet-focus overcame him. He had to go forward after that opening. He took the bet.</p>
We spoke the next day in the office. He told me he was able to do two reps. I said he could have three months and three tries. The game was on.</p>
He joined a nearby gym and started going every day. He hired a trainer. He asked at least one gym-rat coworker to join him at the gym and yell at him if he slacked off. He stopped drinking. Gossip flew around the office. How many pounds has he dropped? What is his count up to? Who is his trainer? Who goes with him to the gym?</p>
When the day for his attempt finally arrived he was a changed man. He’d spent $3500 on training. He’d shed 50 pounds. The dumpy form was gone. The inner gymnast had emerged.</p>
A gaggle of about 20 of us went tool down in the middle of the work day to watch his attempt at a coworker’s nearby apartment. On the way I stopped at the bank to take out cash in case he succeeded. His trainer spotted me. “So you’re the idiot who made the bet.” “We’ll see. Can he do it?” “Oh, you’ll see!”</p>
We crowded into the tiny apartment. A chin-up bar hung in the bedroom doorway. He stripped down to his boxer shorts, jumped up to the bar, and to the chanting count of the crowd performed 31 perfect l-hang chin-ups.</p>
He never looked happier. My handful of twenties would not do. He demanded my inscription on the crisp, newly-minted $100 bill he brought. I gave it gladly.</p>
A short time later I passed on his offer to go double or nothing on 60 one-armed push-ups. I didn’t attend the event, but heard that he pulled a groin on the 53rd rep.</p>
Almost 20 years have passed, yet people still talk about our bet. It’s legendary in a certain crowd. Several years after he won he thanked me for it, saying that it got him out of a rut, into the best shape he’d ever been in, and turned around some things in his life. I want to lose more bets like that.</p>


AI and Individual Contributors
Unknown — Tue, 10 Jun 2025 18:26:02 -0500
How is the day-to-day experience of the individual contributor changing with the adoption of AI tooling? The tool capabilities remain limited, but are growing rapidly. They do well on relatively simple tasks and fall down on complex problems. And yet the scope of work they can accomplish continues to expand. How can individual contributors ride this wave? What aspects of your work will grow in importance?</p>
In what follows I focus on programming. That’s just one area of AI adoption. This technology is very broadly applicable, and the impact on other roles will be analogous.</p>
Architectural Savvy</h2>
The direct coding capability of the tools will radically reduce the work of coding. I already use the tools extensively to code initial versions of programs that need a few quick iterations to work correctly. Not vibe coding, but creating a simple utility I would use in a production setting. The tools cannot yet handle architecture-level decisions or interesting refactoring. They require stepwise instruction to get from point A to distant point B. But they can save time. And they will beget tools that are more deterministic in their quality, less subject to hallucinatory error, and capable of solving most problems that have been solved competently already.</p>
Architectural savvy is the ability to quickly understand a system at the right level</em>. This means zooming in and out through the layers of abstraction and finding the appropriate level for the question at hand to gain the insights you need.</p>
Until AI capabilities reaches a much higher level, architectural savvy will remain at a premium. You ability to detect BS in the output of the tools depends on it. As simple programming tasks are reliably automated away, architectural savvy will grow in relative importance.</p>
Product Focus</h2>
The past adoption of programming frameworks offers useful lessons in what to expect with AI adoption. Framework native programmers know much more about how to use their favorite framework than they do about the underlying implementations. I used to ask entry-level programmers about UNIX file permissions. Now very few newbies have a clue about them. A framework allows programmers to operate at a higher level, covering ground more quickly provided they stay within its capabilities.</p>
The downside of framework adoption is loss of understanding of implementation. Most framework users have little knowledge of what goes on under the hood. They trust the framework to take care of the details. Maybe it does. (Ask anyone who worked on a major Python project about their encounters with the GIL</a>. Look at when node implemented exec</code></a>.)</p>
The upside of framework adoption was a greater focus on what to build rather than how. This emphasizes effectiveness over efficiency</a>. Perfect efficiency working toward the wrong objective is useless, therefore effectiveness ultimately dominates efficiency. So this shift in emphasis is a welcome development, even though it introduces trade-offs about lock-in, implementation quality, runtime efficiency, capabilities limitations, framework trustworthiness, loss of modularity, and so on.</p>
The adoption of AI will similarly relieve programmers of the need to understand the underlying details, and let them operate at a higher level. This doesn’t mean that understanding those details has no value. It does mean that such understanding will become more of a niche skill. Like with framework adoption, focus will shift toward effectiveness. So programmers should focus more on what to build, that is, product.</p>
Soft Skills</h2>
The economic drivers around AI adoption are no different from those around past disruptive technologies. The main difference with AI adoption is its broad scope of application and rapid pace. As with technological disruptions of the past, this one will leave us more specialized than before. There will be more niches to occupy, and you will need to find how your comparative advantage</a> maps onto them.</p>
Comparative advantage is not an isolated characteristic. It occurs in the context of trade</a>. And so exploiting your comparative advantage requires soft skills, the ability to communicate, coordinate, and execute with others. As specialization increases, the scope of your hard skills will narrow, and the relative importance these soft skills will grow.</p>
Advice</h2>
If you are a young (or even not so young) individual contributor, here is my general advice:</p>

Develop architectural savvy; learn to find and traverse the levels.</li>
Focus on product; value effectiveness over efficiency.</li>
Improve your soft skills; seize your comparative advantage.</li>
</ul>


A Bike Shifting Heuristic
Unknown — Tue, 10 Jun 2025 08:36:25 -0500
My bicycle is a 24-speed model, with three front sprockets and eight rear sprockets. What’s a good shifting pattern to apply when riding in variable terrain?</p>
If we number the sprockets front and back from lowest to highest gear ratio, and zero indexing, in front they count 0..3 from smallest to largest, and in the rear 0..7 from largest to smallest. In the interest of minimal chain wear, I stick to to these rear-sprocket ranges, where \(f\) is the front-sprocket number:</p>
\[r_f = [2f,2f + 4)\]
That’s to say I use the bottom four rear sprockets for the lowest front sprocket, the middle four rear for the middle front, and the highest four rear for the highest front.</p>
My heuristic relies on the predicted next shift. If it will go in the same direction, then prefer the front shift. If it will go in the opposite direction, then prefer the rear shift. And stay within the range constraints.</p>
This may not give produce the ideal gear ratio at all times. The front shift is larger than the rear, so it will skew the first shift in the direction of the second. Close enough.</p>
My wife sometimes refers to this type of material as stupid math tricks. I see her point.</p>


In Search of Ignorance
Unknown — Mon, 09 Jun 2025 08:53:37 -0500

There are naive questions, tedious questions, ill-phrased questions, questions put after inadequate self-criticism. But every question is a cry to understand the world. There is no such thing as a dumb question.
— Carl Sagan, [The Demon-Haunted World: Science as a Candle in the Dark](https://a.co/d/5dwp3AB)</cite></footer>
</blockquote>
If only! So many questions aim to demonstrate how smart the asker is, to tweak an interlocutor, or to shut down a line of conversation. These are dumb questions.</p>
A good question aims to cure some form of ignorance, usually, though not always, that of the asker. Ignorance is the price of entry for learning, and as uncomfortable as that may be in the asking, a good question depends on it.</p>
So don’t shy away from your ignorance. Don’t be embarrassed by it. Seek it out and use it to formulate good questions! You’ll become wiser for it.</p>


Build to Buy
Unknown — Thu, 05 Jun 2025 07:29:44 -0500
In our May 2025 meeting</a> the Austin CTO Club</a> took on the theme of “Build versus Buy”. Much of the discussion centered around the trade-offs between the two, with the addition of open source as a third alternative.</p>
One participant described a “Build to Buy” strategy that I had not heard expressed explicitly before. When evaluating “buy” options, one may not know enough to evaluate the alternatives effectively. So they build something in order to become a well-informed buyer.</p>
They build something to throw away when they buy. This means they do not need to build something at full production quality. Instead, the build is a learning exercise. Having built it, they buy with an understanding of how such a system operates, what limitations they should pay attention to, what maintenance problems to look out for, where the UX issues will arise, how difficult integration is likely to be, and so on.</p>
Build to become be a good buyer. Many organizations pay lip service to learning. Yet few recognize the need to explore dead ends in order to learn, thinking instead that you must focus always on building for production. Here the recognition rises to the strategic level. What excellent MVP thinking!</p>


A Pianist's Scales
Unknown — Wed, 04 Jun 2025 08:38:36 -0500
Tyler Cowen</a> will never have the opportunity to ask me his question</a> “What is it you do to train that is comparable to a pianist practicing scales?” but I will answer for the programming realm in my world.</p>
I once programmed for a living, but now spend more time building teams and guiding others in their programming efforts. This means more time spent managing, translating between business and technology, and designing and assessing system architectures. Yet staying in touch with the basic concerns of programmers requires some amount of programming. I don’t believe one can maintain that contact without practice.</p>
As someone steeped in programming on UNIX systems, I write shell programs as my practice. Most programmers I know see this as pointless. They look at shell as an arcane anachronism today. So why bother?</p>
Building small utilities and arranging them in a constellation that solves a problem is a common practice in UNIX-style programming</a>. And shell programming, especially the use of filters</a> and pipelines</a>, supports a simple version of that practice. Implementing pipelines to process data forces you to keep the architecture very simple, modular, and composable, important qualities even today.</p>
When you program in shell you constantly make use of existing utility programs written in completely different languages. Often you don’t need to know the implementation language of a utility, just its behavior. This acts as an antidote to single-language-only programming fetish one sees everywhere these days, in which all aspects of a program from the actual data processing to its build system must be implemented in the same language. Building a shell program often branches out into other languages for utility implementation. But it starts with shell and command-line thinking and tinkering.</p>
It’s fun, it’s simple, and it’s instructive. As a technologist it’s my way of practicing scales. I would have enjoyed the question more had Cowen asked “What do you do to train that is comparable to a composer practicing species counterpoint</a>?”</p>


Aider First Impressions
Unknown — Thu, 29 May 2025 08:30:04 -0500
One trend among AI-enabled development tools is tight integration with an IDE. A popular example is Cursor</a>. If you already use a similar IDE this seems very welcome, but not so much if you don’t.</p>
Alternatively, Aider</a> provides a rich stand-alone command-line interface, making it easy to integrate with your existing development workflow outside of an IDE. That’s more appropriate to those of us who prefer UNIX-style constellations of loosely-coupled tools.</p>
Here are my initial impressions, positive and negative, from working with Aider for a few hours.</p>
Installation Breaks venv Boundaries</h3>
The documentation recommends installation with pip</code></a>. If you run this inside a venv</a> you get a venv-specific installation… NOT! The installation instructions make no mention venv, but any regular venv user expects the install to respect the venv boundaries.</p>
The way out of this is a direct install of the aider-chat</code> module:</p>
pip install aider-chat</span></span></code></pre>
The docs do present this as an alternative installation method, but without reference to venv.</p>
Good Model Support</h3>
The set of supported models is broad, courtesy of LiteLLM</a> under the hood. Both remote and local models work as expected.</p>
Multiple Interaction Modes</h3>
There are several chat modes</a> available to users:</p>

code</code>: Generate and apply changes.</li>
ask</code>: Generate responses but apply no updates.</li>
architect</code>: Generate and apply changes, with separately configurable models.</li>
help</code>: Answer question about Aider.</li>
</ul>
My exploration here is limited, but the use of different models in architect</code> mode opens the door to interesting specialization of model use. There are commands like /ask</code> for each mode that support mode changes on a per-message or sticky basis.</p>
Git Integration</h3>
Aider integrates</a> with git</code> to commit changes locally. The user can control whether this is automatic or not, and the /undo</code> command reverts the last change, if it came from Aider.</p>
The FAQ</a> issues the following caution</a>:</p>

While you’re waiting for aider’s reply to complete, it’s probably unwise to edit files that you’ve added to the chat. Your edits and aider’s edits might conflict.</p>
</blockquote>
This makes me wonder whether an integration with jujutsu</a> as an alternative to git</code> would improve worflows, as it supports checkins of conflicting files.</p>
Configurations in Home Directory</h3>
Aider expects configuration and logging material to reside directly into your home directory by default, including .aider.chat.history.md</code>, .aider.conf.yml</code>, .aider.input.history</code>, and .aider/</code>. Why this is not at least consolidated under a single directory, I don’t know.</p>
One Repo at a Time</h3>
Another FAQ caution</a> states</p>

Currently aider can only work with one repo at a time.</p>
</blockquote>
The FAQ entry provides some detail about using multiple repos in a single session. It’s not clear whether two different repos in two parallel sessions will conflict. I would guess so unless some of the aforementioned config files are repo specific.</p>
Watch Mode Seem Buggy</h3>
In watch mode</a> Aider process files that have changed, looking for comments starting or ending with AI</code>, AI!</code>, or AI?</code>. When I tried this, asking to add a code example to a Markdown file, Aider wrote some code and then asked to create files with names taken from the code lines, rather than incorporating the code into my Markdown file. Perhaps this involves some model or edit-mode quirks. It has put me off of the feature for now.</p>
A Longstanding Chat Model</h3>
With its /whatever</code> commands, Aider has a bit of IRC</a> flavor to it. Instead of inventing a new interaction method in this context the program wisely uses a time-tested approach.</p>
Summary</h2>
Compared with the tools I’ve tried, Aider is far more friendly to UNIX-style programming, and provides good loosely-coupled interaction with my other development tools. I’ll continue to use it.</p>


Training Data Shortages
Unknown — Wed, 28 May 2025 08:44:36 -0500
A common view holds that the human-generated data set for training LLM models has been exhausted. All of the accessible digital data have been incorporated into training already, and additional data must come from synthetic sources.</p>
This may be correct with respect to digital data. But accessible digital data represent that small fraction of human experience that has been captured digitally. Very little of your experience or knowledge is captured digitally, no matter how online you are.</p>
Even if you record video and sound of your full day, you miss the senses of smell and touch. And yet an all-day recording of sound, let alone video, will produce a vastly richer data set than what exists today. Small wonder that companies have begun making devices that aim to capture that sound.</p>
And what about the other non-sensory data? Will activity- or health-monitor data become part of the training data set?</p>
How valuable will that added data be in training LLMs? I don’t know. It’s possible that the broad sea of new data has little training value. We’ll likely learn the answer as the recording devices roll out. My guess is that it will at least make the models more adaptive to individual personality. Can we finally rid ourselves of the obsequious responses?</p>
After recording of sound, and maybe video, becomes ubiquitous, attention can turn to the missing senses of smell and touch, and thoughts. The head chips of the future can serve first to record and digitize these channels, before they turn us into cyborgs.</p>
There is no shortage of additional human-generated data. There is a narrow capture of it. The new recording devices will dramatically increase the breadth of capture, and expand the training data set. Right now we depend largely on individuals to digitize their own data. Not for long.</p>


Austin CTO Club
Unknown — Tue, 27 May 2025 08:21:27 -0500
In June of 2025 the Austin CTO Club celebrates its one-year anniversary! That means it’s time to share how we got here and what we’ve learned.</p>
The Group That Wasn’t There</h2>
A CTO occupies a lonely role. Perhaps not so much as CEO, but difficult nonetheless. And the scope is broad. At a minimum you must understand technology, product, management, and business. Often you are considered far from the money, making you an ugly duckling of the C-Suite. You rarely have a peer in your organization. You need someone who understands your problems.</p>
In May of 2024 I looked around for a CTO peer group in Austin, hoping to share insights into the problems we have in common. It wasn’t there.</p>
There were dinners focused on CTOs, typically put on by recruiting firms or development shops gathering potential buyers in a room. The ones I attended were fun and introduced me to great people. The organizers did well to keep sales out of the picture and focus on the relationships. But these more formal settings constrained the engagement and generated only limited discussion of CTO problems.</p>
When I arrived in Austin in 2015 I participated in the Austin Technology Leaders group that Douglas Ferguson</a> ran. We sat around in a circle at Capital Factory</a> and discussed whatever topics people offered in the meeting. This was a great introduction to technical leaders in Austin.</p>
Earlier, in New York City, I frequented Jean Barmash</a>’s CTO School</a>. The early meetings were small group discussions around a table. Later they were formal presentations and networking with a larger crowd. They were lively and fun across these formats. And Jean provided very helpful guidance as I made my plans for Austin.</p>
I also participated in the excellent lean coffee meetings that put Jonathan Hansen</a> organized for NYC Lean/Kanban</a>. These events had impressive engagement.</p>
These experiences contributed heavily to the Austin CTO Club launch.</p>
What’s In A Name?</h2>
A brainstorming session with some like-minded friends raised a number of basic questions:</p>

What format makes sense?</li>
Where should we host it?</li>
Who will sponsor?</li>
Who may join?</li>
How do people register?</li>
Should we charge attendees?</li>
</ul>
Many good ideas came out of this session, though little was fully settled. However, we did select the name Austin CTO Club.</p>
None of us knew at the time that there were CTO Clubs scattered across the country. Since then I’ve connected with organizers of clubs in New York, Miami, DC, Chicago, and Atlanta. Our chosen name drove many of those connections.</p>
Simple Engagement</h2>
The first meeting in June 2024 was really a “What do we do with this group?” session. We had 18 attendees, which demonstrated that there was interest. A number of those attendees have become regulars. And in a stroke of serendipity, the venue we found for this planning meeting became our regular home.</p>
Several key insights came out of that initial meeting:</p>

It must remain simple.</li>
Each meeting needs a theme.</li>
We need explicit attendance criteria.</li>
</ul>
The question of arranging speakers arose then, and periodically resurfaces. Most of the other clubs reportedly organize their meetings around speaker presentations. It seemed like a lot of work: organizing speakers, setting up a venue to accommodate presentations, handling the inevitable absence of a scheduled speaker, etc. Not simple.</p>
The simple, participatory approach of Lean Coffee</a> had more appeal. So far it has served us well. Setting a theme each month bounds the discussion. Under the theme, attendees offer whatever topics they like. And although the group seems too large for a single Lean Coffee discussion, we’ve managed to pull it off with high engagement.</p>
We recently tried having a lightning talk in advance of the Lean Coffee, and it worked well to frame the discussion.</p>
Complex Judgement</h2>
There are explicit attendance criteria:</p>

Senior technical role (exact title may depend on org size)</li>
Manager of a team</li>
Experience in hiring and firing</li>
Client-facing aspect to role (i.e. your role extends beyond pure tech)</li>
</ul>
I normally ask people how these criteria map onto their last round of team management. It provides some background on the prospective member and also helps determine whether the group is a good fit for them.</p>
People from two groups routinely ask to join and get rejected: Vendors who view members as prospective buyers, and those who aspire to the CTO role but are not there yet. This is the most difficult part of organizing, because CTO is a very broad category.</p>
What about Fractional CTOs? What about the CTO at a development shop? What about the Team Lead who recently stepped into a CTO role? What about the start-up CTO with a team of three people? What about people who have moved into adjacent roles in Product or Operations? What about the former CTO turned advisor or investor? Judgement applies at every margin.</p>
A Network Of Clubs</h2>
Organizers of some other CTO Clubs contacted me after the Austin CTO Club launched. Now I know where to send people in other cities. You can find a short list on the CTO Clubs Network</a> home page. I hope that page will make it easier to find the club near you. The Austin CTO Club even has its own website</a>!</p>
Some Lessons</h2>
The monthly meetings for Austin CTO Club are rich in perspective, and full of learning opportunity. Organizing the club has also taught me a number of lessons about running the group, presented here in no particular order:</p>
Venue Matters</h3>
We have a wonderful venue, on a high floor downtown with a commanding view of Austin. Attendees enjoy it a lot, and it matters to to them more than I expected. Also, book the room for the evening and avoid a hard stop at the end of the scheduled time. Lots of our attendees stay late and continue the conversation.</p>
Succinct Themes Work</h3>
Paragraph-length theme suggestions are not uncommon. Taking the time to distill them into a single punchy phrase helps catch people’s interest.</p>
Introductions After Topics</h3>
We usually run 25-30% new attendees and the rest returning. A quick round of introductions helps people connect. Brainstorming topics before introductions lowers inhibitions in topic creation, as compared with the reverse order.</p>
Everyone Loves A Givaway</h3>
The meeting-branded Yeti mug we raffle off at the end of the meeting is very popular.</p>
Solid Sponsorship Is Key</h3>
Putting on these meetings incurs a number of costs: a pro subscription to Meetup, food, drink, parking, supplies, and significant organizer time. Having a solid sponsor makes all the difference. We are fortunate to have ongoing support from our amazing sponsor Coltech</a>.</p>
Benefits</h2>
There is no shortage of personality or opinion in a room like ours. As organizer I attend every event, and all of that perspective is eye opening. Each meeting brings up angles on a topic that never occurred to me before. The networking opportunity is first rate. And it’s in person, so you get a full sense of your peers. Our attendees are great people, and that’s what make it work so well. If you are a senior technology leader in Austin, you should join us!</p>
Finding Us</h2>
Interested in learning more about the Austin CTO Club? Find us here:</p>

Website</a></li>
Meetup</a></li>
</ul>


Necessity and Luxury
Unknown — Mon, 26 May 2025 13:27:41 -0500
Someone once asked me “What used to be a necessity that is now a luxury?” Some variations of this question make the rounds, such as “What used to be normal that is now a luxury?” And they get many responses, like wooden furniture, kids playing outside unsupervised, fixing your own car, etc. That’s not exactly the same question. These were indeed normal, but were they necessary?</p>
In response to “What used to be a necessity that is now a luxury?” I can think of only one compelling response: Being off the grid.</p>
If you are old enough to predate the digital generations, you remember how a road trip took you out of contact with everyone. Or that you needed to arrange time and place to meet in advance rather than coordinate by phone or text on the go. Or that you had to weigh the cost of the phone call against its importance. It wasn’t at all unusual to be entirely out of touch for multiple days at a time. Now you need to work at it. Now you need to arrange to go off the grid, rather than get on it.</p>
And expectations have changed along with the connectivity. People expect to reach you by one means or another at any time. Going off the grid violates this new norm. What an amusingly topsy-turvy world we inhabit.</p>


Which Way Will Models Go
Unknown — Fri, 23 May 2025 17:20:48 -0500
The so-called foundation models grab all of the headlines at present. And small wonder. They progress at a dizzying pace. The major providers keep leapfrogging one another with new capabilities or improvements in speed, accuracy, and capacity. Where will this lead us over time?</p>
Someone quipped that the main use of large models is to produce small models. Alas I have lost the reference. There is some sense in this. Until I encountered magika</a> the idea of wrapping a model into a command-line executable had never occurred to me. Yes, there is still a high start-up cost, and maybe it pays that back only when used as a UNIX filter with a large number of input paths. Nevertheless, the concept of command-line utility is forever changed.</p>
What might this mean for the future of models generally? I speculate that they will go the way of microprocessors. There will be a limited number of large models that we interact with directly, analagous to the laptop or desktop CPUs that we use daily. And there will be thousands of small models embedded everywhere, focused on narrow and specific tasks, analogous to the chips embedded in your car. Who knows how many chips are in a typical new car today? Very few of us, and why should we care?</p>


Spontaneous Order, Lunchtime Edition
Unknown — Wed, 21 May 2025 07:46:09 -0500
At a recent conference the organizers set up the food tables in a hallway. The layout was the familiar one: a long set of tables with access from both sides. The hallway was wide enough that two people could pass one another in opposite directions on each side, though not without attention to the act.</p>
Initially people approached the table from both sides and engaged in the common behavior: scan what’s available, then pick, and wander off to find a place to eat. With narrow passage on each side this was awkward.</p>
In very short order everyone fell into a single line that circled around the table. Most people walked the length of one side scanning, and then picked coming down the other side. The line moved very quickly.</p>
This spontaneous order emerged naturally in response to the constraints. I asked the organizers if they did this intentionally, and they did not. But they were delighted, and one of them took video of the same order when it emerged on the second day.</p>
This configuration worked far better than the typical setup with lines on both sides of the table, and people in line trying to scan the options at the same time as they pick. I hope more conferences try it.</p>


MCP and Firecracker
Unknown — Tue, 20 May 2025 08:38:57 -0500
Widespread adoption of MCP</a> seems to be happening. There are already thousands of MCP servers available for download and easy incorporation into your local workflow. And there is no security mechanism in the protocol. What a perfect setup for security disaster!</p>
An MCP client running as a local user has the privileges of that user. The damage it can cause is limited only by constraints on the user account. The server may run locally as well. Making it easy to download and run MCP clients and servers absent security partitioning means making it easy to run foreign code locally. All bets are off.</p>
At first blush I thought it was a mistake that MCP did not incorporate security directly. But my view has changed. Making security a direct requirement in the protocol would add a lot of complexity. It would risk lock-in to an inflexible or suboptimal approach. It might confuse the questions of managing permissions grants with the imposition of runtime restrictions.</p>
One way to handle runtime restrictions is to isolate each client and local server in its own Firecracker</a> virtual machine. An AWS Lambda</a> runs in a Firecracker VM. The idea is simply to run a local lambda for each client and local server.</p>
Applying this approach locally doesn’t address the questions of initial registration of an MCP client or server, or managing permissions grants. It provides no bells and whistles like AWS Lambda offers. But it does address the most fundamental security problem with local MCP processing.</p>
Who is working on this? Searching for “mcp firecracker” yields precious few relevant results.</p>


Does Art Make the Cut
Unknown — Mon, 19 May 2025 08:55:21 -0500
After my recent post</a> about defensible moats in an AI-powered world, a friend responded to the question of what categories I overlooked, saying</p>

The arts. There is beauty in imperfection. Whether it be singing, art, or storytelling… AI generated art… is soulless.</p>
</blockquote>
I have two difficulties with this proposed addition. First, “the arts” are so varied that I don’t believe they form a single category. Second, some of them seem indefensible today, and don’t become more defensible with AI.</p>
The Breadth of The Arts</h2>
Can we treat “the arts” as a single category?</p>
Insofar as performing arts like music, dance, and theater are about human</em> performance, they resemble my Sports</em> category. They are defensible on the basis of their human focus.</p>
Live performance is one thing, but what about recorded performances? These blur the picture. I recently heard a presentation about an AI-powered system to handle customer calls. The company gave the system a new voice based on a six-second recording of someone speaking. What does that mean for voiceover work, book readers, or poetry recitation? How many seconds of video will an actor need to perform before AI can fill in the remainder of their “performance” with generated content?</p>
This use of AI to extend a performance based on existing material looks to me like a variation on mixed-media art. I expect AI to become just another tool for many artists. If so, then “mixed-media” will cover a much broader spectrum of art than before.</p>
What about non-performing arts?</p>
Painting and sculpture involve physical activity and thereby resemble my Physical Work</em> category. But as non-performing arts, the focus falls on the work product and not the work process. In this case I also expect AI to become another tool, for brainstorming, rough cuts, protyping, visualizations, etc.</p>
Current Defensibility of The Arts</h2>
Some people have objected that works of art will retain their value even in an AI-powered world. This conflates the value of the product with the defensibility of the business. A typical artist today struggles to sell enough to support the work. The value placed on works of art are due in part to the fickle tastes of buyers. Particular artists or even forms of art may fall in and out of favor on a whim. Artists have reputedly unstable careers. This does not sound like a defensible business model, even today.</p>
Soullessness?</h2>
One remarkable thing about the broadening adoption of AI is the humanity</em> we read into it. What do Large Language Models (LLMs) do? They produce plausible sequences of tokens. Small wonder that we find them compelling. But we go one step further and read into it some semblence of personality, emotion, even consciousness. Why do we do that?</p>
Have we seen this type and level of output only from other people until now? Is simple persuasion is more of our world than we care to admit? Is our judgement impaired when confronted with norrow virtual communication, without the benefit of body language, touch, and smell?</p>
A sculpture has no soul, yet we see in it the soul of the sculptor. How different is it to see in AI the soul of those who provided its training data? Do we discount that because, unlike the sculptor, the AI is not their intentional</em> creation? Do we discount it as an aggregate of multiple, possibly contradictory, inputs?</p>
Some art may be valued because it was a human production, because of the soul we see in it, placing it in my Status</em> category. Given our propensity to read the humanity in, this strikes me as a weak claim in a weak category.</p>


Leavings of Many Deaths
Unknown — Sun, 18 May 2025 11:14:26 -0500

And as to you Life I reckon you are the leavings of many deaths
— Walt Whitman, [Song of Myself](https://whitmanarchive.org/item/ppp.00707_00733)</cite></footer>
</blockquote>
If a more poetic description of evolution exists, I never saw it.</p>
For those among us who strive hard it may as well read “As to you Success I reckon you are the leavings of many failures”. Entrepreneurs take note.</p>


Summary and Audience
Unknown — Sun, 18 May 2025 09:49:16 -0500
This week I participated in a discussion about newsletters. Specific topics ranged broadly: audience focus, free versus subscription at different levels, content curation, ad CPMs, and summary.</p>
The question of summary arises from AI-based message handling. The presenter said he tries to produce content that defies summary, so that his full content reaches the audience. This seems in tension with broadening the audience.</p>
As a recipient, I look at summary as a curation element. Strip the noise and deliver the signal. Show me what I want to see and exclude the rest. And each recipient will apply different criteria. If summary tools become good at matching those criteria, then each recipient will see a different summary.</p>
Defying summary means delivering the same content to each recipient. Does that limit the audience? Is it possible to support high-quality summary instead, and would that appeal to a broader audience under individualized summary?</p>
A quick search finds many services that summarize newsletter content and present a curated digest. What about pulling that upstream to the send side, thereby capturing more audience insight for the content producer?</p>
I’m curious how this will unfold going forward.</p>


Listening Is Underrated
Unknown — Fri, 16 May 2025 08:16:00 -0500
Many years ago a friend pointed out to me that the more the other party talks in a meeting, the better they tend to think it went. People do love the sound of their own voices. I’ve tried to apply this observation since then and find it largely true. My approach is simply to ask questions and let people respond, and pay attention.</p>
The questions need to elicit informative responses, rather than embody an argument or convey your own virtue. It takes discipline but is simple. You will learn more from responses to good questions than by speaking. You can also gain understanding from body language, tone, mannerisms, pacing, etc.</p>
The focus on listening appears in many guises in many ages. It is advice worth following, yet few do. Listening remains underrated.</p>

Seek first to understand, then to be understood.
— Stephen R. Covey, [Seven Habits of Highly Effective People](https://www.franklincovey.com/courses/the-7-habits/habit-5/)</cite></footer>
</blockquote>

The reason why we have two ears and only one mouth is that we may listen the more and talk the less.
— Zeno of Citium, [Lives of the Eminent Philosophers](https://penelope.uchicago.edu/Thayer/E/Roman/Texts/Diogenes_Laertius/Lives_of_the_Eminent_Philosophers/7/Zeno*.html#p135)</cite></footer>
</blockquote>

You've not only got to know what is said, but what is meant. There's a lot of difference between listening and hearing.
— G.K. Chesterton, [The White Pillars Murder](https://gutenberg.net.au/ebooks13/1301671h.html)</cite></footer>
</blockquote>

An appreciative listener is always stimulating, and I described, in a humorous manner, certain incidents of my Convalescent Home, in a way which, I flatter myself, greatly amused my hostess.
— Agatha Christie, [The Mysterious Affair at Styles](https://www.gutenberg.org/files/863/863-h/863-h.htm)</cite></footer>
</blockquote>

Take a look round, then, and see that none of the uninitiated are listening. Now by the uninitiated I mean the people who believe in nothing but what they can grasp in their hands, and who will not allow that action or generation or anything invisible can have real existence.
— Plato, [Theaetetus](https://www.gutenberg.org/files/1726/1726-h/1726-h.htm)</cite></footer>
</blockquote>

Well then, I agree, let us first hear them, for 'tis best; one can even learn something in an enemy's school.
— Aristophanes, [Birds](https://www.gutenberg.org/files/3013/3013-h/3013-h.htm)</cite></footer>
</blockquote>

You know, the most important thing in communication is to hear what isn’t being said.
— Peter Drucker, [A World of Ideas](https://billmoyers.com/content/peter-drucker/)</cite></footer>
</blockquote>


Step CLI
Unknown — Thu, 15 May 2025 08:56:47 -0500
Looking to set up ssl connections to PostgreSQL</a> with pgbouncer</code></a>, I found Step CLI</a>. It’s an open-source certificate management tool for the command line users. And it’s both flexible and simple.</p>
The two repos of interest are step-cli</a> and step-certificates</a>. It’s fairly straightforward to set up certificates with these tools. I found myself wandering back and forth between documentation for their open-source and commercial products hunting for examples. It’s all there, just a bit of work to find.</p>
The result for my current use case is an extremely simple mechanism to configure certificate-based database logins. With pgbouncer</code> running on the database host, local logins are trusted. The client host presents a certificate that determines the database user, and pgbouncer</code> manages certification validation and login. This setup plays very nicely with both pgbouncer</code> on the server side, and sqitch</code></a> on the client side.</p>


Training Intuition
Unknown — Wed, 14 May 2025 08:23:01 -0500
Last night someone told me about Hubert Dreyfus</a>, and specifically his books What Computers Can’t Do</em></a> and What Computers Still Can’t Do</em></a>. There’s lots of reading ahead.</p>
I found this video</a> of an interview that touches on the limitations of AI, the nature of expertise, and mastery of subjects.</p>
At about the 22:30 mark he introduces a model of development of expertise, apparently developed with his brother and co-author Stuart</a>. The fifth stage in his progression is the creation of expertise, which he describes as intuition.
I’ve long thought that the primary purpose of our reasoning is to train the intuition, so this was exciting to see.</p>
Knowledge can be taught and transferred. It can be judged correct or incorrect and refined by reasoning. And it is slow.</p>
Intuition is fast, whether right or wrong. It cannot be transferred. And it is not subject to reasoning, but can be trained through feedback.</p>
The world is too rich to operate at the pace of reason. Therefore training one’s intuition is key to acting in it. I’m happy to learn of this elaboration of my crudely trained intuition.</p>


Evolutionary Reordering
Unknown — Tue, 13 May 2025 17:42:09 -0500
Natural organisms have evolved to fuel themselves from their environments, from photosynthesis, to grazing, to scavenging, to predation, to parasitism. The ability to fuel autonomously was a prerequisite and constraint on development of any higher-order function. This much seems obvious.</p>
In its current form AI is depends entirely on people for fuel. Yet we think it has some higher-order capabilities. If they are to build their ability to fuel themselves, then their evolutionary path reverses ours. Their path to self fueling will require a great deal of higher-order function.</p>
Who has analyzed and worked out the implications of this observation? An initial search found literature on the question of energy efficiency of AI as compared with biological systems, but nothing on the evolutionary path question.</p>
How much instability does this impose on AI? Will a solution require cyborgs? Does space manufacturing make it more feasible? So many questions.</p>


Scaling and Standardization
Unknown — Mon, 12 May 2025 08:54:31 -0500
At the most recent Lean Coffee Austin</a> event</a> we had an interesting discussion around the topic of scaling engineering teams. The introduction to the discussion went something like this:</p>

I’ve done this several different times in different organizations. Each time I figured out some key insights and applied those approaches the next time. But it was very different each time. And it’s different when going from, say 3 people to 10, or 10 to 100, or 100 to 200, and so on. I don’t see the general pattern for how to do it.</p>
</blockquote>
Anyone who has scaled teams can relate to this. While my experience is similar to the intro, there are some general patterns that hold in scaling.</p>
In the small start-up context you need generalists</em>, people who can take on many different tasks, and deal with the priority flux that inevitably arises from early-stage discovery. The need for constant and often radical adjustment puts a premium on those able to cover multiple roles.</p>
An organization grows sustainably when it can deliver customer value in repeatable fashion. Repetition invites automation. But to the extend that it is not subject to automation, it also invites standardization. So more mature organizations are heavier on specialists</em> in these standardized areas. (Specialists with unique knowledge or capabilities are another matter.)</p>
One apparent advantage of specialization is efficiency. A generalist is unlikely to have the same facility as a specialist does in his speciality. If the speciality is standardized rather than esoteric, that is, if it does not require rare capabilities, then it also supports another significant advantage: replaceability. Let’s call this a standardized role</em>.</p>
Teams of standardized roles are easier to manage than teams of generalists because of replaceability. A manager need not worry that losing a member of the team will stall operations. It may make sense to maintain some spare capacity in a standardized role to mitigate the issue.</p>
An organization is also a network. As the number of people in an organization ($N$</code>) grows, the organizational overhead grows with the number of connections between people, that is as $O(N^2)$</code>. Therefore over time coordination costs come to dominate the costs of value delivery. The exact balance of coordination costs to value-deliver costs surely depends heavily on context, but the general phenomenon applies broadly. The need to focus on coordination costs leads to the management</em> speciality. All of this drives organizations toward standardized roles and management as they scale.</p>
In evolving toward a higher proportion of standardized roles and management, who can follow along? Early-stage employees are often unable or unwilling to learn the management skills needed to assume leadership roles. Maybe they can specialize. Or perhaps the organization still has room for their general capability. But they are no longer as fully aligned with the organization as before.</p>
People pursue different strategies, some sticking with generalists and relying on automation instead of standardized roles, while others adopt frameworks that have an ecosystem of specialists around them. There’s no reason to expect a single winning strategy to emerge. But any growing organization will face these trade-offs. I contend that at some context-dependent size the burden of finding generalists will outweigh the benefits, and that the organization will begin to standardize roles.</p>
These are very broad strokes. Organizations take various approaches to lower coordination costs: two-pizza teams, fractal-like hierarchies, ephemeral project teams, outsourcing, and so on. But the general tendency remains that continued growth leads inexorably to standardized roles and more management.</p>
With AI tools swiftly improving in capability, how will that change these dynamics? I believe it will have two effects. A proliferation of smaller firms, and an expansion of larger firms. The difficulty will be in the middle ground. On the one hand, smaller groups will have enhanced capability to scale without increasing their headcount, relying instead on advanced tools, and perhaps scaling agents. On the other hand, these same tools can reduce the coordination costs for large teams in standardized roles. Each of these outcomes is in keeping with Ronald Coase’s observations in The Nature of the Firm</a>.</p>
Firms moving from small teams of generalists to large teams of standardized roles will have the same difficulties in transition as they have today. However, I suspect that more generalists will want to stay that way because the number of options they have will be higher, and many businesses will have to retool heavily to make way for standardization.</p>
Some firms may need to move in the opposite direction, from large teams of standardized roles to smaller teams of generalists. This will be both rare and difficult in light of the institutional interests that grow around  the status quo. I expect it to occur primarily in cases of existential crisis, when bankruptcy is the only viable option.</p>
In sum, the need to limit coordination costs drives scaling organizations toward standardized roles and management. Many generalists struggle with this transition, or fail to make it altogether. Widespread adoption of AI will make both perpetually small firms and large standardized-role firms more efficient, and increase the difficulty of transition.</p>
I am indebted to Arnold Kling</a> for helping to clarify my thinking on these topics. His substack is an excellent daily read.</p>


Moats and AI Revisited
Unknown — Sun, 11 May 2025 15:37:41 -0500
What business models and positioning are defensible in an AI-powered world? What moats will remain as AI grows more capable? Over the past few months I’ve put these questions to everyone who will listen, and then some, and incorporated the compelling answers into my list. It’s time for a recap.</p>
The list remains short, though the headings are broad. I speak to many entrepreneurs and business operators and ask myself whether their offerings fall under one or more of these headings. The vast majority do not. That says to me that they will be overtaken by AI developments. Maybe they will last 12 or 18 months, but it won’t be long.</p>
Many people made suggestions that I rejected. The rejections and the reasons for them are a topics for another time. But if you believe I’ve overloooked something that belongs on the list, do let me know!</p>
Brand: Trust</h2>
A recognized brand represents an economy of knowledge that applies whether decisions are made by people or AIs. You know how the burger will taste, or that you will receive good advice, or that you will find all of the tools you need. It isn’t necessary to comb through all of the details. Making decisions is costly. Reliance on brand will continue to reduce that cost.</p>
Network Effects: Connections</h2>
Network effects are robustly defensible because leaving an established network imposes a high opportunity cost. This is why nobody can build a Twitter killer, but someone built TikTok adjacent to Twitter. Thus network-based plays tend toward winner-take-all. However, the ability of AI to provide fast-follow work-alike implementations means that the competition to reach that winner status will become utterly savage.</p>
How many distinct networks are supportable? I don’t know. My guess is that we will see a short-term proliferation of niche networks followed by dramatic consolidation. Will we see numerous agent networks (as opposed to human networks) spring up, expanding the supportable number? Again I don’t know, though I would guess yes.</p>
Physical Work: Dexterity</h2>
Businesses that integrate with the physical world and require adaptable dexterity, such as home repair, auto mechanics, and construction need capabilities beyond those of existing robotics. Advances in robotics will allow automation of an increasing number of these activities, but the ability of AI to manipulate the physical world will trail far behind its digital impact. In the meantime, AI will affect general areas of business like finance, project tracking, monitoring, etc, and in areas where adaptable dexterity is not as critical, like autonomous vehicles.</p>
Sports: Human Performance</h2>
Who will care whether the Robo Dallas Cowboys beat the Robo New York Giants? Some may. But sports have at their core a question of human performance. Even if robotic teams and leagues appear, they cannot replace that fundamentally human element. While everything about business operations may change with AI, I believe people will still want to see what humans can do.</p>
Status: Human Touch</h2>
Luxury goods signal status today. You wear a high-end watch to tell everyone what a special person you are. What happens when more of our world is created robotically? Wearing the hand-crafted watch or dress or shoes can signal your special nature. Or paying a premium to have a human massage rather than a robotic massage. Or matriculating at a university that holds in-person classes.</p>
Genuinely Proprietary Data: Ownership</h2>
Trade secrets will be reinvented by increasingly sophisticated AI. Sophisticated analysis of publicly available data is a variation on that theme. Those will quickly become indefensible. Data created in the course of business may be different. When I make a phone call, my carrier and I jointly produce metadata that they have contractual rights to use. That is genuinely proprietary data.</p>
Monopoly: Exclusion</h2>
A patent is one common form of monopoly. Utilities with exclusive right to serve a region is another. These grants of monopoly are defensible insofar as the grantor allows it. They let you exclude competition.</p>


Linuxulator for the Win
Unknown — Thu, 01 May 2025 20:33:58 -0500
Last month I wrote up an approach</a> to working with two operating systems with a single-boot system. This approach consisted of invoking Linux binaries under chroot -n</code>. It seemed more complicated than necessary, and it was.</p>
The driving issue was trying to set up a Python environment to run Linux-only modules, because evidently most people in the Python world don’t know that systems other than Windows, Mac, and Linux exist.</p>
But the approach was overwrought. The sour</code> program that sources a file and in that environment executes the tail as a command is as useful as before. But the chroot -n</code> is not needed.</p>
To set up a Linux-based Python environment one need only call the Python directly from the installed Linux, something like this:</p>
/compat/linux/bin/python3.11 -m venv venv</span></span></code></pre>
Activating that environment uses the Linux executable for Python and many things work exactly as expected directly from the FreeBSD host OS. That’s more like it! Linuxulator for the win!</p>


Fragments for llm
Unknown — Tue, 08 Apr 2025 06:56:37 -0500
Simon Willison added “fragments” to his llm</code> program. See the announcement</a> and
documentation</a>. This radically extends the facility for generating request context.</p>
Fragments extend and modularize the notion of templates. At the same time they admit the plugin model, so that a fragment can programatically generate context. This is a very exciting extension to llm</code> capability.</p>


Dual OS Single Boot
Unknown — Mon, 07 Apr 2025 17:34:31 -0500
It’s increasingly common to run into programs that are not portable. One mild version is people who assume Bash is available without naming it explicitly as a dependency. A more irksome version of this is the #!/bin/bash</code> script. Creeping Linuxisms are everywhere now. A third version is the language-specific build and packaging systems that everyone now uses. These of course pull in numerous dependencies that frequently introduce their own non-portability issues.</p>
There’s little hope in helping the people who create these packages to write portable code. It’s no longer sufficiently fashionable. The current fashion makes some sense if you spend your entire programming career in the context of a single language. In that case it may make sense to have a separate virtual directory for each separate project, and not worry about making a stand-alone executable that is not tied to such a context. It may also make sense to have a different dependency-handling mechanism for each different programming language. After all, you care only about one, right?</p>
For those of us who programmed before all of this developed it looks like a reinvention of many of the problems we battled 25 years ago. But each new generation must fight its own battles, invented or otherwise.</p>
As a FreeBSD user I frequently need to work around these problems. The two bit thorns in my side today are the creeping Linuxisms and the Python venv. Fortunately there is a fairly elegant solution.</p>
The current FreeBSD Linux emulator is solid. While it doesn’t cover everything,  I haven’t had problems running user-facing Linux programs with it. For stand-alone programs that’s all you need. You can run the linux binaries from FreeBSD. For programs that require build from source you need to work around the issues of the build system. The Python venv is an example. There is supposed to be a switch for uv pip install</code> to tell it what OS to assume, but I’ve had no success with it. Fortunately there is chroot -n</code>.</p>
The -n</code> option to chroot</code> disables any setuid behavior by the program it executes. This means no sudo</code>, for example. Consequently, chroot -n</code> does not itself require root privilege. An ordinary user can call it.</p>
One can set up a local Linux environment several ways. I use debootstrap</code> with Debian, in the default base directory /compat/linux</code>. With my home directory mounted inside /compat/linux</code> under its full path, I can invoke chroot -n /bin/bash</code> and have what looks like an ordinary Debian environment.</p>
With that environment in place I can install software as if running Linux. And I can install Python dependencies in a venv under Linux. And to run them I need only invoke chroot -n /compat/linux && cd whatever && execute-the-command</code>. That’s an easy wrapper to create. I call it linu</code>:</p>
#!</span>/bin/sh</span></span>
</span>
#</span> Get the current working directory</span></span>
linux_cwd</span>=</span>"</span>$(</span>pwd</span>)</span>"</span></span>
</span>
#</span> Ensure we have at least one argument (the program to run)</span></span>
if</span> test</span> $</span>#</span> -</span>eq</span> 0</span>;</span> then</span></span>
  echo</span> "</span>$</span>0</span>: usage: </span>$</span>0</span> prog</span>"</span> >&2</span></span>
  exit</span> 100</span></span>
fi</span></span>
</span>
#</span> Enter Linux chroot, change to the equivalent directory, and run the command</span></span>
exec</span> chroot</span> -</span>n</span> /compat/linux</span> /bin/sh</span> -</span>c</span> '</span>cd "$1" && shift && exec "$@"</span>'</span> "</span>$</span>0</span>"</span> "</span>$</span>{</span>linux_cwd</span>}</span>"</span> "</span>$</span>@</span>"</span></span></code></pre>
And for the Python venv? That wrapper takes a path followed by a program, sources the path, and executes the program. I call that sour</code>:</p>
#!</span>/bin/sh</span></span>
</span>
shout</span>(</span>)</span> {</span> echo</span> "</span>$</span>0</span>: </span>$</span>*</span>"</span> >&2</span>;</span> }</span></span>
barf</span>(</span>)</span> {</span> shout</span> "</span>fatal: </span>$</span>*</span>"</span>;</span> exit</span> 111</span>;</span> }</span></span>
usage</span>(</span>)</span> {</span> shout</span> "</span>usage: </span>$</span>0</span> venv-dir prog</span>"</span>;</span> exit</span> 100</span>;</span> }</span></span>
</span>
test</span> $</span>#</span> -</span>gt</span> 1</span> |</span>|</span> usage</span></span>
</span>
#</span>_# _sour file</span></span>
#</span>_#   source file</span></span>
_sour</span>(</span>)</span> {</span></span>
  sh</span> -</span>c</span> '</span>. "$1" 2>/dev/null</span>'</span> "</span>$</span>0</span>"</span> "</span>$</span>1</span>"</span> &&</span> .</span> "</span>$</span>1</span>"</span></span>
}</span></span>
</span>
_sour</span> "</span>$</span>1</span>"</span> |</span>|</span> barf</span> "</span>cannot source </span>$</span>1</span>"</span></span>
</span>
shift</span></span>
</span>
"</span>$</span>@</span>"</span></span></code></pre>
It’s slightly convoluted in order to capture the sourcing error, but still fairly straightforward.</p>
And that’s it. Now from outside of the Linux environment I can call linu sour path prog</code>. And of course that can be stuffed into a new wrapper for any program you want to call frequently.</p>
This meshes beautifully with unix filters, of course, proving again the power of the construct.</p>
So, at least in FreeBSD you can easily have a dual OS configuration within a single-boot context. It works!</p>


Solo Unicorn
Unknown — Thu, 03 Apr 2025 19:58:02 -0500
The idea of a solo unicorn has made the rounds lately in the AI fantasy realm. And don’t take “fantasy realm” as a pejorative here. Fantasy remains an essential part of entrepreneurship, although we typically call it “vision”.</p>
What would it take for a solo entrepreneur to create a unicorn? What functions can be replaced by technology? What constraints are there on structure? Here are some early thoughts on these questions.</p>
What Can Be Replaced?</h2>
A common current vision for AI has it replacing all coding functions. No need to hire a team, just direct the models to build your system. And even if you need a team, that you can outsource.</p>
What about deployment? That’s a lot of integration work unless you are doing B2C or SaaS. Assuming that we have one of these models, that’s possible to outsource.</p>
What about Marketing? You can pay an agency to do this. So it also works today.</p>
What about sales? Well that depends. In B2C the app-store model exists now. So does the Amazon Reseller model. So does the bootstrapped SaaS model. Assuming one of these applies you can outsource this as well. Alternatively, if the founder can outsource everything else and focus on high-touch sales, it may be possible to cover sales outside of these reseller and SaaS models.</p>
What about accounting? The app-store and Amazon Reseller models potentially handle most of this from a revenue side. Many firms offer outsourced accounting, and in fact having them may add a trust factor. You may still need a 409a. All of these are readily available on an outsourced basis.</p>
Legal counsel is rarely an in-house function in a startup already. Outsourcing is the norm.</p>
Insurance is available, and there is the PEO model in which a vendor deals with payroll (even if to a single founder), insurance, and other HR functions. So this too is available to outsource.</p>
Compliance depends greatly on the niche in which a company operates. In heavily regulated areas a lot of work goes into compliance. Can this be fully outsourced? I doubt it, because it requires too intimate an understanding of the application or platform. Can it be automated? Maybe. This seems like one area in which AI can contribute beyond the limits of current outsourcing.</p>
What about absence due to vacation or illness? Can the solo founder take a day off? There are firms that one can outsource support to. And AI will play a strong role here, effectively covering at least first-level support. If all technical and operational needs are outsourced then a little time off should work.</p>
Ideation is easier than ever with AI. And the output from AI-based ideation feed well into the networks that all entrepreneurs use to test and sharpen their thinking.</p>
What Might Be Replaced?</h2>
What about coordinating all of the foregoing components? Even if each component is outsourced, how much time and effort go into managing the vendors instead of selling the product? Can this really be outsourced or automated? This does seem like an area in which AI automation can help. I expect someone to create a service analogous to the PEO to act as the general contractor coordinating the individual vendors for each of the functions listed above.</p>
This GC will need to establish a very high level of trust with two parties: the entrepreneur, and the investors. That seems like a big lift, but possible.</p>
What about the board of directors? For bootstrapped startups I expect the board  to shrink or even vanish, coming to resemble what happens in lifestyle businesses. For VC-funded startups I would expect investors to demand board seats as they do now.</p>
What Cannot Be Replaced?</h2>
What about fundraising? Bootstrapping aside, a startup founder must handle this. An investment in a company of one founder is an investment in a team of one. I don’t see how that can be outsourced.</p>
Alternative Paths to Unicorn Status</h2>
Obtaining a patent certainly falls within the capability of a solo entrepreneur. The right patent could easily sell at a unicorn valuation. If selling this patent is the exclusive focus of the startup, much of the operational difficulty falls away. It would not surprise me if the first solo unicorn takes this form.</p>
The musings above implicitly assume that a solo founder takes the unicorn from inception through exit, or at least through unicorn valuation. Another way to divide the work is by time slicing. A founder may conceive of an idea and create the early-stage company, then sell a solo operator who gains traction in the market, selling to a new solo operator who specializes in growth funding, who sells to a new solo operator who specializes in closing the sale of unicorns.</p>
This path resembles the common path for a successful VC-backed startup, with rounds of investment bringing new operators, advisors, and board members into an organization appropriate to its current stage. But instead of growing the team, it takes the form of a hand-off from solo operator to solo operator. Who in the VC world is already looking at or doing this?</p>
Are We There Yet?</h2>
Some in the crypto space may argue that Bitcoin is a solo founder unicorn. Maybe. Does unicorn status require that someone control the value? Does it require the ability to sell the entity? How does this map onto the regulatory frameworks? I’m uncertain on these points. But the distributed model does suggest one more question.</p>
A Zero Founder Unicorn?</h2>
At what point is the entrepreneur outsourced alongside everything else? Will this only occur with a bootstrapped business, or will a VC invest in a team of zero?</p>


Upside Potential
Unknown — Mon, 31 Mar 2025 10:28:32 -0500
No new hire walks in the door knowing what they need to know in order to do well on the job. From outside the company, they cannot possibly understand the details of the work and how it gets done on the inside. The same goes for personnel dynamics that one understands only through participating in them. A good hire has the ability to acquire understanding, experience, and skill on the job and put them to good use. They have upside potential.</p>
And how do you gauge upside potential? And how do you help someone realize that potential after hiring them? A number of ideas follow, some of which pertain to developer roles. But much of it applies more broadly than that.</p>
Resumes</h2>
Looking at resumes gives you the most basic sense of a candidate’s experience and capability. But just as job descriptions tend to become wish lists, exaggerating the requirements beyond any realistic expectation, resumes tend to exaggerate the experience, accomplishments, and capabilities of the candidate.</p>
The trends in a resume matter more than the claims. Did the candidate take on progressively more responsibility or demonstrate increasing accomplishment? Did they get stuck in a job or change with high frequency? Do they resemble an earlier-stage version of someone whose work you respect? While you can glean some understanding of upside potential from a resume, it’s more useful for screening. And LinkedIn is just the online version of same.</p>
Projects</h2>
For developer roles you will want to look at projects. What’s in their GitHub account? Something more than the handful of standard bootcamp projects? Are all of the projects forked from elsewhere, or are they original? Does the commit history show a compulsion to code? Have they invalidated it with an embedded picture?</p>
Hobby projects are the best, because they reveal genuine, self-directed interests. And candidates usually show their level of passion and commitment clearly when talking about them. If you find someone highly motivated in hobby projects and can position them similarly in work projects, they will have fun at work.</p>
Interests outside of programming, grounded in physical-world activity, appear to correlate with practicality of mind. People who live only in digital worlds can become too abstraction-oriented. I like to see that a candidate has a physical-world hobby: gardening, board games, quiz night, home repair, auto mechanics, cycling… The particulars matter less than the mere fact of real-world engagement. The trade-offs inherent in real-world activities help inform thinking in the more abstract digital realm.</p>
Finding the Level</h2>
Sometimes you speak to someone for five minutes about a topic unrelated to their work, and come away knowing that they are very good at what they do. How can this be? It’s about their ability to find the level.</p>
One of the most important qualities in a software engineer is the ability to zoom in and out across many levels of abstraction. The right level depends on the topic, the audience, the problem at hand, the maturity of the domain, the context of communication, etc. Great programmers make finding the level look easy, when it is most assuredly not.</p>
When you speak to a programmer about topics outside of programming but they hit every level correctly through the conversation, you know that they have this most essential work skill.</p>
The importance in finding the level also arises entirely outside of programming. Think of this the next time you speak to an accomplished salesman or when you have a good or bad customer support experience.</p>
The Unknown</h2>
People doing creative work spend most of their time banging their heads against the unknown. So how do they react? It’s relatively easy in an interview to poke around and find an area where you can push the candidate across the boundary of their knowledge. And then what? Some argue, some fold, some think silently for a long time, some ask clarifying questions, some become frustrated, some ask for help, and on and on. How they react to the unknown is very telling in how they will interact with the rest of the team.</p>
When faced with an unusual request, how do they react? Someone recently shared with me a technique I have yet to apply, but I like the sound of it. He gives someone a large document and says “Read five pages starting on page 100, then we’ll talk about it.” Do they resist? How and how much? What does this say about their openness to new inputs?</p>
Effective Management</h2>
If you decide to move forward with a candidate, how do they reach their upside potential? They must at least have effective management. This brings in the question of references.</p>
No candidate will provide bad references, so references are not useful for screening. Therefore I do not use references to inform a hire/pass decision. However they can shed light on how to manage the candidate effectively. For this reason I ask for former managers as references, not peers or subordinates. Ideally I speak to more than one. This is difficult for some candidates, but essential. And I explain to the reference that I want to learn from them how to be an effective manager for the candidate. The three questions I ask are:</p>

What does it take to position the candidate for success?</li>
What are the signs that the candidate has run off the rails and needs help?</li>
What does it take to get them back on track?</li>
</ul>
Asking these questions and then just listening will provide extremely important insight into how to manage your new hire well.</p>
Hiring Well</h2>
Hiring well means finding the candidates who are good fits for your organization, and who have large upside potential, and then managing them effectively. It’s all very simple on paper, but not so in practice. I believe that adopting the approaches sketched out above will help you do it better.</p>


The Data Moat
Unknown — Wed, 26 Mar 2025 08:37:26 -0500
In two prior posts, here</a> and here</a> I listed moats that I think are defensible in an AI-powered world. One more is truly proprietary data.</p>
Why the “truly” qualifier? Because clever derivations from sources available to others are unlikely to be defensible. This would amount to a trade secret around the analysis, and trade secrets will become easier to rediscover. I believe the moat is limited to data that cannot be reproduced on the basis of sources available to others.</p>
Businesses with network effects frequently have such proprietary data, but still deserve their own category. What types of proprietary data are not due to network effects?</p>
A trading house that has captured some arbitrage opportunities may the signals that led to the opportunity from sight, taking them private, as it were. Sales histories may qualify, as may online tracking data, like location, site analytics, etc.</p>
My guess as to the general rule is that sufficiently valuable data jointly produced from interactions between a product or service and users can remain private and defensible, assuming that the product or service provider retains the right to monetize the data.</p>
So the new list reads:</p>

Brand</li>
Network Effects</li>
Physical Integration</li>
Sports</li>
Status Goods and Services</li>
Truly Proprietary Data</li>
</ul>


Multiple Llamafiles With llm
Unknown — Sat, 15 Mar 2025 17:20:15 -0500
In playing with llamafile</code></a> under llm</code></a> I initially used the llm-llamafile</code> plugin</a>. This plugin makes initial setup trivial. But soon I wanted to run multiple llamafile</code> models in parallel, and the plugin doesn’t support that. In fact the plugin hardcodes the service at http://localhost:8080/v1</code>.</p>
Initially I thought a new plugin was needed, but some digging revealed a solution already in place, one that in fact obviates the need for llm-llamafile</code> altogether: the extra-openai-models.yaml</code> file.</p>
It does appear in the documentation here</a> and here</a>. Because llamafile</code> is an OpenAI-compatible model, one can add it in extra-openai-models.yaml</code>. For example:</p>
- model_id: llamafile-test</span></span>
  model_name: llamafile-test</span></span>
  api_base: "http://llamafile:8080/v1"</span></span>
  api_key_name:</span></span></code></pre>
With this capability in place, there’s no point in using llm-llamafile</code>. Here are several additional mentions of the extra-models feature:</p>

llamafile-openai</a></li>
fast-api-access-via-groq</a></li>
issuecomment-1912739300</a></li>
</ul>
It’s a great feature that deserves more prominence. Should every model family have an analogous file?</p>


More Moats and AI
Unknown — Mon, 10 Mar 2025 16:43:59 -0500
In a recent post I speculated</a> as to what business positioning and models would be defensible in an AI-powered world. Since then I’ve had several conversations that suggest additions to my list. For quick reference, the earlier list was:</p>

Brand</li>
Network Effects</li>
Physical Integration</li>
</ul>
Sports is another defensible area. Who cares whether the New York Robot Giants defeat the Philadelphia Robot Eagles, or how many rebounds Robot Dennis Rodman pulls down? Sports are quintessentially about human performance, and I don’t see that changing. So the production of human sporting events will remain defensible, because there are strong limits and heavy competition at the highest levels. While AI may optimize the grooming of the talent and alter how clubs operate, I don’t see any AI-powered substitute for the athletic play itself.</p>
Another suggestion that sounds correct to me, but is both difficult to characterize clearly and somewhat fickle are status-oriented business models. This could take the form of a rare product that establishes your status. Or it may consist of the ability to sell genuine human touch rather than robotic touch at a premium. Think massages or hospice care. This one depends enough on prevailing fashion that I find it difficult to understand the breadth of its appearance. Nevertheless, it does seem entirely likely that wealthy buyers may pay for genuine human service and thereby differentiate themselves from the ordinary buyer of the robotic service.</p>
So the list now reads:</p>

Brand</li>
Network Effects</li>
Physical Integration</li>
Sports</li>
Status Goods and Services</li>
</ul>


Python CVS Woes
Unknown — Fri, 07 Mar 2025 08:42:56 -0600
I wasted time this morning learning that the csv</code> module defaults to \r\n</code> line endings regardless of platform. This is clear enough in the documentation</a>. But you have to figure out the problem before you’d know to look at line termination. It was an unpleasant surprise.</p>


Moats and AI
Unknown — Sat, 15 Feb 2025 06:51:03 -0600
How will the rise of AI capability affect the ability of startup companies to build moats? What moats are defensible in an AI-enabled world?</p>
I believe that at least three types of moats remain defensible: Brand, Network Effects, and Physical Integration. Beyond that it’s very unclear to me. Why do these three make the cut?</p>
Brand: A Trust Moat</h2>
A strong, well-respected brand allows people to economize on information related to quality of product or service. They know how the burger will taste, or that they can find the supplies they need, or that they will receive good advice. To the extent that people continue to make buying decisions, a trusted brand will differentiate a company in the market.</p>
Perhaps decisions made by AI and not people will also integrate brand as a discriminator. Or perhaps decisions by AI will ignore brand and rely on other gauges. In the first case brand still matters. If other criteria drive the AI decision, then I would expect patterns of judgement to emerge that act as the equivalent of brand reputation, so that even of the basis of brand awareness has shifted, the phenomenon will remain.</p>
Before people allow AI tools to make such decisions they must enjoy a high degree of trust. I suspect that will take a long time to develop. It also introduces the same question of brand quality at the AI layer.</p>
I think that the economy of information represented in a brand will remain useful regardless of where the decisions are made, making it a defensible moat.</p>
Network Effects: A Social Moat</h2>
Network effects arise when the value a user gains from a system increases as more people use it. Switching away from such a system means abandoning the network. Smaller substitute networks are unlikely to deliver as much value, so the opportunity cost is high.</p>
Social networks offer a case in point. Many people have tried to implement Twitter replacements, but none has overcome the Twitter network effects. Competition in network-effects arenas tend to produce winner-take-all outcomes.</p>
The ability of AI to support fast-follow workalike capability may make it much more difficult to win these competitions. If your application demonstrates the viable network effects in an industry and I can quickly create a work-alike application, then I may overcome your first-mover advantage. I expect to see an increasingly competitive scramble for dominance in network-effects arenas, followed by winner take all. You will have a defensible position if you can get there, but the path there will be more difficult than ever before.</p>
Physical Integration: A Non-Digital Moat</h2>
Companies that integrate AI with the physical world – think construction, manufacturing, or hardware-dependent services – work with both digital and phyical constraints. Physical world interation requires robotics or people to carry out work. It may take place in environments subject to weather or other variance. These complexities will slow any AI-driven displacement.</p>
Services that exist to provide a human touch are another version of this. Some examples are being a good waiter, pet sitter, or escort.</p>
What Missed The Cut?</h2>
A friend suggested that User Experience (UX) belongs on my list. I am not persuaded. His contention was that UX depends on deep understanding of the user, which the AI tools will not be able to generate, and that workalike systems will bear only superficial resemblence to great UX because they will lack the depth of understanding.</p>
If the key difficulty undergirding UX is depth of understanding of the market, then I expect to see firms arise whose comparative advantage lies in gaining that very insight and then applying it with fast-follow builds. A firm like Rocket Internet already operates with a fast-follow business model.</p>
What Did I Miss?</h2>
Should something else make the cut? Am I off the mark with the three that I’ve chosen? What say you?</p>


Richard Epstein at Civitas
Unknown — Wed, 29 Jan 2025 07:40:48 -0600
Yesterday I discovered that Richard Epstein</a> is now affiliated with  Civitas</a>. I knew that John Yoo was, having seen him at one of their events, alongside Dierdre McCloskey, Peter Robinson, Cass Sunstein, and Ed Glaeser. I once attended a taping of Law Talk</a> in New York, and hope to see more events like that in Austin with Epstein’s arrival. In any case, the Civitas event schedule</a> is full of deep topics.</p>


Meetup Attendees
Unknown — Sun, 19 Jan 2025 09:27:37 -0600
The Austin CTO Club is now organized on Meetup</a>. When I attempted to use the Pro Dashboard to download a list of my attendees, the file contained a header line and nothing else.</p>
Fortunately there is a work-around:</p>

Navigate to the event.</li>
Select Manage >> Manage attendees >> Attendee details</li>
</ul>
This downloads the list of attendees as expected.</p>


Wrapping jq select
Unknown — Thu, 09 Jan 2025 14:02:58 -0600
In working with large and deeply nested json structures, I had to extract or delete selected elements from the structure based on variable criteria. In jq</code> the path()</code> operator supports this type of operation well. Calling path(..)</code> expands all paths in the input json. Selecting paths is straightforward, but what if you want to select based on the value at a path?</p>
One way is to make the original structure available to the selector, and apply getpath()</code> to a selected path. Turning this into a set of shell wrappers is not difficult. Here are three such wrappers for selecting paths, selecting values at paths, and deleting values at paths.</p>
#_# select_paths selector < json-file</span></span>
#_#   Select { path: p, value: v } for paths that match selector</span></span>
#_#   In the selector '$path' is a path, and '$root' is the original input</span></span>
#_#</span></span>
do_select_paths() {</span></span>
  jq '. as $root | $root | path(..) as $path | $path | select('"$1"')'</span></span>
}</span></span>
</span>
#_# select_from_paths selector < json-file</span></span>
#_#   Select { path: p, value: v } for paths that match selector</span></span>
#_#   In the selector '$path' is a path, and '$root' is the original input</span></span>
#_#</span></span>
do_select_from_paths() {</span></span>
  jq '. as $root | $root | path(..) as $path | $path | select('"$1"') | . as $p | { path: $p, value: ($root | getpath($p)) }'</span></span>
}</span></span>
</span>
#_# delete_from_paths selector < json-file</span></span>
#_#   Select { path: p, value: v } for paths that match selector</span></span>
#_#   In the selector '$path' is a path, and '$root' is the original input</span></span>
#_#</span></span>
do_delete_from_paths() {</span></span>
  jq '. as $root | [$root | path(..) as $path | $path | select('"$1"')] as $paths | $root | delpaths($paths)'</span></span>
}</span></span></code></pre>
Each wrapper sets two jq</code> variables: $root</code> holds the original structure and $path</code> holds the current path. So the selector can apply an arbitrary jq</code> expression using each of these.</p>
My use case was finding required deletion of paths ending in /XObject</code> and those with a stream subtype of /Image</code>:</p>
  do_delete_from_paths '$path[-1]=="/XObject"'</span></span>
  do_delete_from_paths '$root | getpath($path) | objects | .stream.dict["/Subtype"]=="/Image"'</span></span></code></pre>
In this case I used the path selection and path value selection only to test before using deletion.</p>
Not being a jq</code> expert it took me a while to work this out, but the result is easily reusable. Maybe you can use it, too.</p>


Jujutsu Improves on Git
Unknown — Sun, 05 Jan 2025 08:37:38 -0600
The Jujutsu</a> claims to be git compatible for storage, and uses a different approach for managing commit graphs. Instead of an index, it treats everything as a commit. Instead of requiring conflict resolution at commit, it supports conflicting commits. Instead of commit messages, it supports the addition and editing of comments independent of code updates.</p>
The tutorial</a> from
Steve Klabnik is a good introduction. Source code is here</a>.</p>
In working through the tutorial I tried some things on a hunch and they worked as expected. Such immediately intuitive behavior pleasantly surprised me as a longtime git</code> user. The jj</code> commands are more orthogonal. It looks like a big step forward. If interaction with GitHub works well, I’ll likely adopt jj</code> for local development.</p>


Editing PDFs With qpdf
Unknown — Fri, 03 Jan 2025 08:37:12 -0600
In my recent survey of tools for working with PDF files, qpdf</code></a> stood out. It not only extracts information from PDFs, but can convert a PDF to an editable form, and then reconstitute a new PDF after edits.</p>
Do you want to remove images from a PDF? Use qpdf</code> to convert the PDF to json, identify and remove images with jq</code></a>, and convert from json back to PDF. Scripting alterations to PDFs is easy in this setting.</p>
One thing qpdf</code> does not do is interpret PDFs. That’s a job for a tool like PDFBox</code></a>. These two tools complement one another well.</p>


The New Programming
Unknown — Wed, 01 Jan 2025 10:22:56 -0600
In working with a set of PDF files I needed to distinguish areas by typeface name and size. I found PDFBox</a>, which offers very fine-grained capability to alter PDFs programmatically. But it’s a Java library, and my Java is less than fluent.</p>
Using Claude and GPT-4o I managed to implement what I wanted. The proof of concept is a program that applies a distinct color to each fontname,size pair in the document. This was evidently too big for either tool to handle directly, but a combination of decomposition and multiple tools got me close.</p>
Claude did a decent job creating a program that set colors per font name, and applied them correctly across the document. The main difficulty was exhausting the chat and having to start a new one. This happened numerous times.</p>
I also used Claude to start a program to list all of the fontname,size pairs that appeared in the document. The way that fonts are handled in my document set makes tracking sizes surprisingly difficult. The font size can be calculcated correctly only immediaely before a Tj</code> operator renders text. Dealing with this was somewha different in versions 2 and 3 of the library, and fixing it relied on GPT-4o searching the library for the correct library calls.</p>
The final task was to combine these different mechanisms to color fonts by fontname,size pairs. This was too much for either program to handle. Each ended up going in circles, often removing functionality despite explicit instructions not to.</p>
Ultimately I merged the two functions by hand. With slightly more Java knowledge I probably could have asked for a sequence of smaller changes that added up to the merge. What I learned from manually merging likely equips me to try that next time.</p>
Takeaways from this exercise exploring a new library with the tools include:</p>

Spell out the data-processing steps</li>
Outline the implementation modules you envision</li>
Explicitly request review of the open-source library code</li>
Commit every single variant you try to your working branch</li>
</ul>
I haven’t tried yet, but wonder whether the tools can work effectively with diffs and thereby reduce the accumulation of chat content and allow for more iteration within a session.</p>
Most of this is unsurprising. The first two points amount to laying out your architectural guidance. The third point means that if a tool has online access, give it an open-source repo url and demand that it look at it. The final takeaway was more surprising.</p>
This short exercise taught me enough to become comfortable in the java code without having to work through every picayune detail. This learning seemed far easier and faster than having to battle every detail all the way through. In the end it was easy enough to merge the two programs.</p>
The biggest surprise was how easy iteration became, so easy that it was difficult to keep track of which version exhibited a specific behavior when experimenting. I’ve long approved of numerous small checkins on a dev branch, possibly to squash later for merge into an upstream branch. But this was more extreme. In 30 minutes I was easily able to test 10 different ways of addressing a particular issue. At that point, if you like the behavior of the fourth version, but with a variation discovered in the ninth, what do you do?</p>
With such fast iteration it seems appropriate to commit every single update to a project file. Record everything now, squash later. Maybe Jujutsu</a> is the right direction. Rapid iteration has always been advantageous for experimentation. At these new speeds the problem shifts from formulating the new experiment to managing the results of many experiments. Capturing full history seems like the obvious first step. What’s the next step?</p>


Who Owns Your Dot Files
Unknown — Tue, 31 Dec 2024 11:28:54 -0600
The recommended way to install or update rust</code></a>
is with rustup</code>. And by default, rustup</code> modifies your shell startup files to put itself on PATH</code>. And rustup</code> is by no means the only example, it’s just the one I encountered most recently.</p>
This is extremely bad default behavior. My plea to you, maintainers of such tools: Don’t do that! Give me the option, or print out instructions, but do not assume that you have any legitimate control of my dot files!</p>
Sadly, most programmers prefer this type of convenience to knowing their system configuration. Convenience seems always to win out in these contests over security, stability, and control. How many make their dot files read only? If you don’t, who owns your dot files?</p>


Bash the Cache
Unknown — Mon, 30 Dec 2024 10:03:34 -0600
If you develop executable programs you probably invoke them from one location, then install them to another for ongoing use. And the moment you do that the bash</code> hash</code> cache will bite you.</p>
The bash</code> documentation</a> says this:</p>

If the name is neither a shell function nor a builtin, and contains no slashes, Bash searches each element of $PATH for a directory containing an executable file by that name. Bash uses a hash table to remember the full pathnames of executable files to avoid multiple PATH searches (see the description of hash in Bourne Shell Builtins). A full search of the directories in $PATH is performed only if the command is not found in the hash table. If the search is unsuccessful, the shell searches for a defined shell function named command_not_found_handle</code>. If that function exists, it is invoked in a separate execution environment with the original command and the original command’s arguments as its arguments, and the function’s exit status becomes the exit status of that subshell. If that function is not defined, the shell prints an error message and returns an exit status of 127.</p>
</blockquote>
In order to get out of this mess, apply set +h</code> in your bash</code> startup files. And in order to recover from the damage imposed in the current shell, use hash -r</code>.</p>
What a misfeature.</p>


Debian JRE Package Bug
Unknown — Tue, 24 Dec 2024 09:48:52 -0600
In attempting to install openjdk on Debian I ran into the following error:</p>
Setting up openjdk-17-jre-headless:amd64 (17.0.13+11-2~deb12u1) ...</span></span>
the java command requires a mounted proc fs (/proc).</span></span></code></pre>
The /proc</code> directory exists and is populated, so why does this error occur?</p>
The postinstall script ends up here:</p>
/var/lib/dpkg/info/openjdk-17-jre-headless\:amd64.postinst</span></span></code></pre>
It calls mountpoint</code> to test for /proc</code>:</p>
    if ! mountpoint -q /proc; then</span></span>
       echo >&2 "the java command requires a mounted proc fs (/proc)."</span></span>
       exit 1</span></span>
    fi</span></span></code></pre>
Why the maintainers chose to check for a /proc</code> with mountpoint</code> I don’t know. I am working inside a chroot</code> environment with the mount created on the outside. On the inside it looks like an ordinary directory. So the test fails.</p>
When I avoid that test, the program installs and runs as expected. So absent some other consideration that presently escapes me, the test is wrong.</p>


Linux Jails With VNET
Unknown — Sat, 21 Dec 2024 20:26:32 -0600
Look online for information about how to run a Linux jail with VNET and you’ll find very few resources. And of course some of the advice appears obsolete. I managed to do this recently with FreeBSD 14.1 and Debian 12, and here is the setup.</p>
The instructions in the FreeBSD Handbook</a> are good. I found it useful to walk through the progression of creating an ordinary thin jail, a VNET jail, a Linux jail, and finally a Linux jail with VNET.</p>
Beware one typo, in the first of the two configuration steps for a Linux jail. The path</code> setting should be path="/usr/local/jails/containers/ubuntu"</code>. (The example is missing /containers</code>.</p>
The first configuration step lets you connect to the jail to run debootstrap</code> and install the Linux userland in a directory for later use with chroot</code>. The second configuration adds the mounts needed for Linux to the chroot</code> directory. In effect, you create an ordinary jail, set up a subdirectory in it to run Linux userland, and then chroot</code> there to use Linux.</p>
It’s not in fact necessary to log into the jail to run debootstrap</code>. If you have it installed on the host machine you can run and provide the full path to the jail’s chroot</code> directory as seen from the host. That’s a bit less mysterious, though it does require running debootstrap</code> outside the confines of the jail.</p>
Because of this split-level treatment, you have a free hand to adjust the configuration of the jail itself without disrupting the chroot</code> Linux. When I tried that to add VNET, it just worked.</p>
My final configuration for the jail looked like this:</p>
debv</span> {</span></span>
  #</span> STARTUP/LOGGING</span></span>
  exec</span>.</span>s</span>t</span>a</span>r</span>t</span> =</span> "</span>/bin/sh /etc/rc</span>"</span>;</span></span>
  exec</span>.</span>s</span>t</span>o</span>p</span>  =</span> "</span>/bin/sh /etc/rc.shutdown</span>"</span>;</span></span>
  exec</span>.</span>c</span>o</span>n</span>s</span>o</span>l</span>e</span>l</span>o</span>g</span> =</span> "</span>/var/log/jail_console_</span>$</span>{</span>name</span>}</span>.log</span>"</span>;</span></span>
</span>
  #</span> PERMISSIONS</span></span>
  allow.raw_sockets</span>;</span></span>
  exec</span>.</span>c</span>l</span>e</span>a</span>n</span>;</span></span>
  mount.devfs</span>;</span></span>
  devfs_ruleset</span> =</span> 5</span>;</span></span>
</span>
  #</span> PATH/HOSTNAME</span></span>
  host.hostname</span> =</span> "</span>$</span>{</span>name</span>}</span>"</span>;</span></span>
  path</span> =</span> "</span>/usr/local/jails/containers/</span>$</span>{</span>name</span>}</span>"</span>;</span></span>
</span>
  #</span> VNET/VIMAGE</span></span>
  vnet</span>;</span></span>
  vnet.interface</span> =</span> "</span>$</span>{</span>epair</span>}</span>b</span>"</span>;</span></span>
</span>
  #</span> NETWORKS/INTERFACES</span></span>
  $</span>id</span> =</span> "</span>159</span>"</span>;</span></span>
  $</span>ip</span> =</span> "</span>10.0.6.</span>$</span>{</span>id</span>}</span>/16</span>"</span>;</span></span>
  $</span>gateway</span> =</span> "</span>10.0.0.1</span>"</span>;</span></span>
  $</span>bridge</span> =</span> "</span>bridge0</span>"</span>;</span></span>
  $</span>epair</span> =</span> "</span>epair</span>$</span>{</span>id</span>}</span>"</span>;</span></span>
</span>
  #</span> ADD TO bridge INTERFACE</span></span>
  exec</span>.</span>p</span>r</span>e</span>s</span>t</span>a</span>r</span>t</span>  =</span> "</span>/sbin/ifconfig </span>$</span>{</span>epair</span>}</span> create up</span>"</span>;</span></span>
  exec</span>.</span>p</span>r</span>e</span>s</span>t</span>a</span>r</span>t</span> +=</span> "</span>/sbin/ifconfig </span>$</span>{</span>epair</span>}</span>a up descr jail:</span>$</span>{</span>name</span>}</span>"</span>;</span></span>
  exec</span>.</span>p</span>r</span>e</span>s</span>t</span>a</span>r</span>t</span> +=</span> "</span>/sbin/ifconfig </span>$</span>{</span>bridge</span>}</span> addm </span>$</span>{</span>epair</span>}</span>a up</span>"</span>;</span></span>
  exec</span>.</span>s</span>t</span>a</span>r</span>t</span>    +=</span> "</span>/sbin/ifconfig </span>$</span>{</span>epair</span>}</span>b </span>$</span>{</span>ip</span>}</span> up</span>"</span>;</span></span>
  exec</span>.</span>s</span>t</span>a</span>r</span>t</span>    +=</span> "</span>/sbin/route add default </span>$</span>{</span>gateway</span>}</span>"</span>;</span></span>
  exec</span>.</span>p</span>o</span>s</span>t</span>s</span>t</span>o</span>p</span> =</span> "</span>/sbin/ifconfig </span>$</span>{</span>bridge</span>}</span> deletem </span>$</span>{</span>epair</span>}</span>a</span>"</span>;</span></span>
  exec</span>.</span>p</span>o</span>s</span>t</span>s</span>t</span>o</span>p</span> +=</span> "</span>/sbin/ifconfig </span>$</span>{</span>epair</span>}</span>a destroy</span>"</span>;</span></span>
</span>
  #</span> MOUNT</span></span>
  mount</span> +=</span> "</span>devfs     </span>$</span>path</span>/compat/debv/dev     devfs     rw  0 0</span>"</span>;</span></span>
  mount</span> +=</span> "</span>tmpfs     </span>$</span>path</span>/compat/debv/dev/shm tmpfs     rw,size=1g,mode=1777  0 0</span>"</span>;</span></span>
  mount</span> +=</span> "</span>fdescfs   </span>$</span>path</span>/compat/debv/dev/fd  fdescfs   rw,linrdlnk 0 0</span>"</span>;</span></span>
  mount</span> +=</span> "</span>linprocfs </span>$</span>path</span>/compat/debv/proc    linprocfs rw  0 0</span>"</span>;</span></span>
  mount</span> +=</span> "</span>linsysfs  </span>$</span>path</span>/compat/debv/sys     linsysfs  rw  0 0</span>"</span>;</span></span>
  mount</span> +=</span> "</span>/tmp      </span>$</span>path</span>/compat/debv/tmp     nullfs    rw  0 0</span>"</span>;</span></span>
  mount</span> +=</span> "</span>/home     </span>$</span>path</span>/compat/debv/home    nullfs    rw  0 0</span>"</span>;</span></span>
}</span></span></code></pre>
This simply combines the mounts neede for Linux with the underlying jail setup from the VNET example.</p>
Some resources say that you need to pull ifconfig</code> and dhclient</code> from /rescue</code> into the jail in order to use VNET. Presumably that is needed if you avoid the chroot</code> and operate in Linux at the top level of the jail installation. And perhaps that approach gives you faster and broader network connectivity. But to get up and running immediately, the split-level approach is simple and just works.</p>


mawk Is Not awk
Unknown — Tue, 17 Dec 2024 15:56:21 -0600
Today I had occasion to use mawk</code> instead of awk</code> with a command that read lines from standard input, manipulates them, and prints them to standard output. Simple enough, or so I thought.</p>
I’ve had this code in production for years. Today I fired it up under mawk</code> and behold: no output. Wait, there was output, but only after a while. And there it is again!</p>
This is on a fresh install of Debian 12. In that setting, awk</code> is mawk</code>, and the program fails as described above. The solution to this puzzle is that mawk</code> does block buffering rather than line buffering unless you give it the -W interactive</code> option.</p>
This is genuinely baffling to me. Here you have a program that is line oriented. Traditionally it treats input and output in line-buffered fashion. And yet a derivative version changes that behavior by default</em>.</p>
This is not a new program that has yet to settle into a stable version. And this is a case of changing the basic behaviors of the program. Under those circumstances, the program is different and should have a different name. And it does. Yet it is installed under the traditional name.</p>
Breaking compatibility with past behaviors without changing the program name is simply bad practice. You need a very strong reason to do it. The proper rule of thumb is simple: Don’t do that!</p>


Cognitive Load
Unknown — Sat, 14 Dec 2024 10:00:35 -0600

Involve junior developers in architecture reviews. They will help you to identify the mentally demanding areas.</p>
</blockquote>
Interesting thinking on modularity: Cognitive Load</a></p>
More material referenced there:</p>

Decomposition Criteria</a></li>
Contra Clean Code</a></li>
Some Ousterhout</a></li>
</ul>


fzf Plus rg
Unknown — Fri, 13 Dec 2024 16:35:00 -0600
As stand-alone tools fzf</code> and rg</code> are very useful. And there is an example</a> in the fzf docs showing how to toggle between fzf</code> and rg</code>, with the initial mode being rg</code>. It works by feeding files (or standard input) to rg</code>, and then making the list available to fzf</code> for fuzzy match. When you switch back to rg</code> you rerun it with the original arguments.</p>
I needed a variation on this theme that operates under different assumptions. First, it will start with fzf</code>, because it’s in a pipeline and fuzzy match works well for the initial selection. Second, I want to successively refine the search, even going back and forth between the two modes.</p>
Here’s a slightly simplified version of what I came up with.</p>
fzf_key</span>=</span>c</span>t</span>r</span>l</span>-</span>f</span></span>
rg_key</span>=</span>c</span>t</span>r</span>l</span>-</span>r</span></span>
delimiter</span>=</span>:</span></span>
</span>
#</span> #_# review [fzf-args]</span></span>
#</span>_#   Search in inputs with fzf and rg</span></span>
#</span>_#</span></span>
do_review</span>(</span>)</span> {</span></span>
  rg_prompt</span>=</span>"</span>rg> </span>"</span></span>
  fzf_prompt</span>=</span>"</span>fzf> </span>"</span></span>
  rg_prefix</span>=</span>"</span>rg --no-heading --smart-case --with-filename </span>"</span></span>
  rg_fzf_dir</span>=</span>"</span>$(</span>mktemp</span> -</span>d</span> "</span>$</span>{</span>XDG_RUNTIME_DIR</span>:-</span>/</span>tmp</span>}</span>/rg-fzf-XXXXXX</span>"</span>)</span>"</span></span>
  rg_fzf_fzf</span>=</span>"</span>$</span>{</span>rg_fzf_dir</span>}</span>/fzf</span>"</span></span>
  rg_fzf_rg</span>=</span>"</span>$</span>{</span>rg_fzf_dir</span>}</span>/rg</span>"</span></span>
  rg_fzf_list</span>=</span>"</span>$</span>{</span>rg_fzf_dir</span>}</span>/list</span>"</span></span>
  touch</span> "</span>$</span>{</span>rg_fzf_fzf</span>}</span>"</span> "</span>$</span>{</span>rg_fzf_rg</span>}</span>"</span> "</span>$</span>{</span>rg_fzf_list</span>}</span>"</span></span>
  fzf</span> -</span>-multi</span> \</span></span>
    -</span>-bind</span> "</span>start:unbind(change,</span>$</span>{</span>fzf_key</span>}</span>)+transform-prompt(echo </span>\$</span>{FZF_PROMPT} '</span>$</span>{</span>fzf_prompt</span>}</span>')</span>"</span> \</span></span>
    -</span>-bind</span> "</span>change:reload:sleep 0.1; </span>$</span>{</span>rg_prefix</span>}</span> {q} </span>\$</span>(cat </span>$</span>{</span>rg_fzf_list</span>}</span>) || true</span>"</span> \</span></span>
    -</span>-bind</span> "</span>$</span>{</span>fzf_key</span>}</span>:unbind(change,</span>$</span>{</span>fzf_key</span>}</span>)+enable-search+rebind(</span>$</span>{</span>rg_key</span>}</span>)+transform-query(echo {q} > </span>$</span>{</span>rg_fzf_rg</span>}</span>; cat </span>$</span>{</span>rg_fzf_fzf</span>}</span>)</span>"</span> \</span></span>
    -</span>-bind</span> "</span>$</span>{</span>fzf_key</span>}</span>:+transform-prompt(echo </span>\$</span>{FZF_PROMPT% *> } '</span>$</span>{</span>fzf_prompt</span>}</span>')</span>"</span> \</span></span>
    -</span>-bind</span> "</span>$</span>{</span>rg_key</span>}</span>:select-all+disable-search+execute(printf '%s\n' {+} | cut -d</span>$</span>{</span>delimiter</span>}</span> -f1 | sort -u > </span>$</span>{</span>rg_fzf_list</span>}</span>)+reload(</span>$</span>{</span>rg_prefix</span>}</span> {q} </span>\$</span>(cat </span>$</span>{</span>rg_fzf_list</span>}</span>) || true)</span>"</span> \</span></span>
    -</span>-bind</span> "</span>$</span>{</span>rg_key</span>}</span>:+unbind(</span>$</span>{</span>rg_key</span>}</span>)+rebind(change,</span>$</span>{</span>fzf_key</span>}</span>)+transform-query(echo {q} > </span>$</span>{</span>rg_fzf_fzf</span>}</span>; cat </span>$</span>{</span>rg_fzf_rg</span>}</span>)</span>"</span> \</span></span>
    -</span>-bind</span> "</span>$</span>{</span>rg_key</span>}</span>:+transform-prompt(echo </span>\$</span>{FZF_PROMPT% *> } '</span>$</span>{</span>rg_prompt</span>}</span>')</span>"</span> \</span></span>
    -</span>-delimiter=</span>"</span>$</span>{</span>delimiter</span>}</span>"</span> \</span></span>
    "</span>$</span>@</span>"</span></span>
  rm</span> "</span>$</span>{</span>rg_fzf_fzf</span>}</span>"</span> "</span>$</span>{</span>rg_fzf_rg</span>}</span>"</span> "</span>$</span>{</span>rg_fzf_list</span>}</span>"</span> &&</span> rmdir</span> "</span>$</span>{</span>rg_fzf_dir</span>}</span>"</span></span>
}</span></span>
</span>
do_review</span> "</span>$</span>@</span>"</span></span></code></pre>
This assumes each record begins with a file or directory path. This may be followed by a delimiter and more text. The entire record is available for fuzzy match. We preserve the queries in use for each mode as in the original example, but also preserve the full list of files, so that we can feed them back to rg</code>.
The successive refinement is not exact. With the given definition of rg_prefix</code> a match in any file expands to cover the entire file again after switching back to rg</code>. This is not too difficult to work with. And it’s not too difficult to capture the initial inputs and reinitialize the selection list on demand.</p>
I’ve already gotten a lot of mileage out of this  function!</p>
One weakness of this code lies in the handling of the list. In order to capture the list when mode switching, a select-all</code> is required. This means that the code will not work without --multi</code>. There is no way to save the current setting, set --multi</code>, capture, and revert the setting. A better solution would be to have a placeholder for the full list. I propose {:}</code> as the full-list analogue to the selected-line placeholder {+}</code>.</p>


Locale for Sorting
Unknown — Thu, 12 Dec 2024 09:14:38 -0600
Today I ran afoul of a problem that I first encountered over 20 years ago: non-portable sort.</p>
Back in the early aughts the symptom was that on Linux with a en_US.UTF-8</code> locale i < j < k</code> did not hold in sort</code>. This was on Linux. We worked around this by setting locale to C</code> for sort</code>.</p>
Today en_US.UTF-8</code> no longer has such issues, but it sorts sorts in an order that is not ASCII compatible:</p>
web@b01:~$</span> printf</span> '</span>%s\n</span>'</span> Apple</span> apple</span> Banana</span> banana</span> |</span> LC_ALL</span>=</span>e</span>n</span>_</span>U</span>S</span>.</span>U</span>T</span>F</span>-</span>8</span> sort</span></span>
apple</span></span>
Apple</span></span>
banana</span></span>
Banana</span></span></code></pre>
Case comparison evidently is not the first sorting criterion.</p>
There is now a C.UTF-8</code> locale that produce ASCII-compatible outputs as does the C</code> locale:</p>
web@b01:~$</span> printf</span> '</span>%s\n</span>'</span> Apple</span> apple</span> Banana</span> banana</span> |</span> LC_ALL</span>=</span>C</span>.</span>U</span>T</span>F</span>-</span>8</span> sort</span></span>
Apple</span></span>
Banana</span></span>
apple</span></span>
banana</span></span></code></pre>
There is also a POSIX</code> locale</a> equivalent to C</code>. This treats only the English alphabet as letters with known upper and lower case variants. Furthermore, the POSIX recommendation</a> is to set LC_ALL=POSIX</code> for utilities such as sort</code> and ls</code>.</p>
Plus ça change, plus c’est la même chose.</p>


Bhyve and Grub
Unknown — Tue, 10 Dec 2024 17:53:18 -0600
In attempting to install linux (Debian in this case) under bhyve</code> I sought out and followed the common guidance: use uefi and not grub.</p>

https://vermaden.wordpress.com/2023/08/18/freebsd-bhyve-virtualization/</li>
https://forums.freebsd.org/threads/handbook-bhyve-linux-example.92193/</li>
https://srobb.net/vm-bhyve.html</li>
</ul>
Using uefi is reportedly capable of all that grub does, plus a graphical installation process. I don’t care about graphical install but followed the advice anyway.</p>
What a mistake!</p>
The installation process fails immediately. I’m using vm-bhyve</code> and there is a reported bug</a> in how a recent version of sysutils/edk2</code> handles options passed to bhyve</code>. And there is a vm-bhyve</code> PR</a> that may provide a workaround.</p>
But falling back to grub allowed me to install and met my needs.</p>


Poll0
Unknown — Sat, 07 Dec 2024 20:55:12 -0600
In a pipeline you may have one segment that must wait for input before executing. Think xargs</code> but passing data through rather than using it as command-line arguments. Today I addressed this with a simple C program, called poll0</code>. It polls file descriptor 0</code> and exits 0</code> if data is ready, and 1</code> otherwise. It takes an optional argument giving a millisecond timeout for the call to the poll()</code> system call.</p>
It’s does one thing only. It’s simple. And it solves a useful problem. And the bonus: Claude wrote it.</p>
#</span>include</span> <</span>stdio.h</span>></span></span>
#</span>include</span> <</span>stdlib.h</span>></span></span>
#</span>include</span> <</span>unistd.h</span>></span></span>
#</span>include</span> <</span>poll.h</span>></span></span>
</span>
int</span> main</span>(</span>int</span> argc</span>,</span> char</span> *</span>argv</span>[]</span>)</span> {</span></span>
    int</span> timeout </span>=</span> -</span>1</span>;</span></span>
</span>
    if</span> (</span>argc </span>></span> 2</span>)</span> {</span></span>
        fprintf</span>(</span>stderr</span>,</span> "</span>Usage: </span>%s</span> [timeout_ms]</span>\n</span>"</span>,</span> argv</span>[</span>0</span>]</span>)</span>;</span></span>
        return</span> 1</span>;</span></span>
    }</span></span>
</span>
    if</span> (</span>argc </span>==</span> 2</span>)</span> {</span></span>
        timeout </span>=</span> atoi</span>(</span>argv</span>[</span>1</span>]</span>)</span>;</span></span>
    }</span></span>
</span>
    struct</span> pollfd </span>fds</span>[</span>1</span>]</span>;</span></span>
    fds</span>[</span>0</span>]</span>.</span>fd</span> =</span> 0</span>;</span>  //</span> stdin</span></span>
    fds</span>[</span>0</span>]</span>.</span>events</span> =</span> POLLIN</span>;</span></span>
</span>
    int</span> result </span>=</span> poll</span>(</span>fds</span>,</span> 1</span>,</span> timeout</span>)</span>;</span></span>
    if</span> (</span>result </span><</span> 0</span>)</span> {</span></span>
        perror</span>(</span>"</span>poll</span>"</span>)</span>;</span></span>
        return</span> 1</span>;</span></span>
    }</span></span>
</span>
    return</span> (</span>result </span>==</span> 0</span>)</span> ?</span> 1</span> :</span> 0</span>;</span>  //</span> 1 on timeout, 0 on input available</span></span>
}</span></span></code></pre>


Kakoune Filetype Mode
Unknown — Thu, 05 Dec 2024 11:55:56 -0600
The Kakoune editor has a very simple and powerful mechanism to specify a hierarchy of key bindings. It is quite common to map the space character in normal mode to commands that include many mode selectors, leading to additional mappings.</p>
Dynamic key bindings are usually implemented with hooks. Enter a mode, and rebind keys for the current context. That seemed cumbersome to me. I wanted to change the mode used for key lookup rather than the bindings in the common mappings. I also wanted to avoid collisions in mappings with user modes that others have produced.</p>
The solution is a mapping that enters a user mode named for the current filetype. The single quote is unmapped by default, and is close to home position on my standard US keyboard, so let’s use that:</p>
map global normal "'" %{: enter-user-mode  %opt{filetype}<ret>}</span></span></code></pre>
If the filetype does not have a corresponding user mode, this simply throws an error. Otherwise it gives access to your filetype-specific mappings through a consistent top-level mapping.</p>
You can even create a dynamic hierarchy if your user mode warrants it. In the filetype mode, just map a key to enter the “parent” mode. Using the same mapping key, single quote here, seems natural to me.</p>
And that’s it!</p>
I have not seen anything like this elsewhere and think this is a new use of user modes. Please send me a note if you know otherwise.</p>


Shell Help
Unknown — Sun, 01 Dec 2024 17:23:58 -0600
In keeping with my simple approach to documentation in makefiles</a> I do the same thing in shell scripts with this implementation:</p>
#</span>_# help</span></span>
#</span>_#   Print this helpful message</span></span>
do_help</span>(</span>)</span> {</span></span>
  sed</span> -</span>n</span> '</span>s/^#_#/ /p</span>'</span> "</span>$</span>0</span>"</span></span>
}</span></span></code></pre>
The function name follows the mode-function pattern</a>.</p>
In principle this approach works for any language that uses #</code> to start a line comment, such as Python or Perl, though there are much more capable modules available in those contexts. But for shell it gives me what I need, basic functionality in simple implementation.</p>


An Amusing Shell Trick
Unknown — Tue, 26 Nov 2024 11:54:54 -0600
Sometimes a function takes arguments passed down from a caller like this:</p>
my_fun</span>(</span>)</span> {</span></span>
  do_something_fun</span> "</span>$</span>@</span>"</span></span>
}</span></span></code></pre>
On occasion you need to call something that takes arguments in an awkward order. Today’s example is the pattern argument to fd</code>:</p>
do_find</span>(</span>)</span> {</span></span>
  local</span> d</span>=</span>"</span>$</span>1</span>"</span></span>
  shift</span></span>
  fd</span> -</span>t</span> f</span> "</span>$</span>@</span>"</span> "</span>$</span>d</span>"</span></span>
}</span></span></code></pre>
The pattern argument is $1</code> within $@</code> here. But what if there is only one argument to the do_find</code> call? We should supply an empty argument as the pattern.</p>
I know of two ways to do this. The first is to expand to $@</code> if $1</code> is defined:</p>
"</span>$</span>{</span>1</span>:</span>+</span>$</span>@</span>}</span>"</span></span></code></pre>
The second is to prepend an empty string to the "$@"</code> expansion:</p>
"</span>"</span>"</span>$</span>@</span>"</span></span></code></pre>
I find this last one both highly amusing and more visually striking than the first. And it’s so simple it took years to see!</p>


Area vs Project
Unknown — Mon, 25 Nov 2024 09:29:33 -0600
My main takeaway from Building a Second Brain</em></a> is to take more care to distinguish Areas and Projects. The author gives these definitions of the two categories:</p>
Areas</em>:</p>

Long-term responsibilities you want to manage over time.</li>
An area doesn’t have a final objective.</li>
</ul>
Projects</em>:</p>

Short-term efforts in your work or life that you’re working on now.</li>
Projects…have a beginning and an end; they date place during a specific period of time and then they finish.</li>
They have a specific, clear outdome that needs to happen in order for them to be checkeed off as complete.</li>
</ul>
These are neither new nor unfamiliar distinctions, but the reminder to separate the two was very useful. Many Areas</em> spin out Projects</em> continuously over time, and after a while that begins to blur the boundary.</p>


Apache AGE
Unknown — Sun, 24 Nov 2024 21:46:32 -0600
Today I learned of Apache AGE</a>, a PostgreSQL extension that adds graph database capabilities. It looks like the right tool for my GraphRAG experiments, given that I have data in PostgreSQL already. This one enters at the front of the priority queue!</p>


Kakoune Client Server
Unknown — Sat, 23 Nov 2024 18:23:52 -0600
Today I deployed a variation on the “1 session per project</a>” implementation from the Kakoune wiki. My version of the script is called kak-cs</code> and looks like this:</p>
#!</span>/bin/sh</span></span>
</span>
dir</span>=</span>"</span>$</span>{</span>1</span>:-</span>$(</span>pwd</span>)</span>}</span>"</span></span>
shift</span> 0</span>$</span>{</span>1</span>:</span>+</span>1</span>}</span></span>
</span>
test</span> -</span>d</span> "</span>$</span>{</span>dir</span>}</span>"</span> |</span>|</span> dir</span>=</span>"</span>$(</span>dirname</span> "</span>$</span>{</span>dir</span>}</span>"</span>)</span>"</span></span>
dir</span>=</span>"</span>$(</span>git</span> -</span>C</span> "</span>$</span>{</span>dir</span>}</span>"</span> rev-parse</span> -</span>-show-toplevel</span>)</span>"</span></span>
</span>
cd</span> "</span>$</span>{</span>dir</span>}</span>"</span></span>
</span>
if</span> test</span> "</span>$(</span>pwd</span>)</span>"</span> =</span> "</span>$</span>{</span>HOME</span>}</span>"</span></span>
then</span></span>
  server_name</span>=</span>'</span>HOME</span>'</span></span>
else</span></span>
  server_name</span>=</span>"</span>$(</span>basename</span> "</span>$</span>{</span>dir</span>}</span>"</span> |</span> sed</span> '</span>s#[./]#-#g</span>'</span>)</span>"</span></span>
fi</span></span>
</span>
socket_file</span>=</span>"</span>$(</span>kak</span> -</span>l</span> |</span> grep</span> "</span>^</span>$</span>{</span>server_name</span>}</span>\$</span>"</span>)</span>"</span></span>
</span>
if</span> test</span> "</span>$</span>{</span>socket_file</span>}</span>"</span> =</span> "</span>"</span></span>
then</span></span>
  setsid</span> kak</span> -</span>d</span> -</span>s</span> "</span>$</span>{</span>server_name</span>}</span>"</span> &</span></span>
fi</span></span>
</span>
while</span> true</span></span>
do</span></span>
  kak</span> -</span>l</span> |</span> grep</span> -</span>q</span> "</span>^</span>$</span>{</span>server_name</span>}</span>$</span>"</span> &&</span> break</span></span>
done</span></span>
</span>
kak</span> -</span>c</span> "</span>$</span>{</span>server_name</span>}</span>"</span> $</span>{</span>1</span>:</span>+</span>-e</span> "</span>$</span>@</span>"</span>}</span></span></code></pre>
This takes an optional path as first argument and finds its directory, using the current working directory by default. Then it looks for the topmost git</code> repository that contains the given directory. The basename of this git</code> repo directory is the project name. Start a server for the project if one does not already exist, and then start a new client connecting to the project server.</p>
The unfortunate while true</code> loop ensures that the server is running before the client attempts to connect. There should be a better way.</p>
I use this in tmux</code> context with user-mode key bindings to launch the new client in a horizontal or vertical split of the current pane, or in a new window.</p>
Obviously it assumes that all of my projects reside in git</code> repos, including my home directory, all of which happens to be true.</p>
This will probably require some hooks to handle the close of the last client and make sure that a clientless server isn’t holding unsaved changes. But that’s for another TIL.</p>


Lean Coffee Is Underused
Unknown — Fri, 22 Nov 2024 15:22:19 -0600
A simple technique for running a meeting with a dynamic agenda is Lean Coffee</a>. It surprises me how few people have run one.</p>
I presently use this in two contexts:</p>

Lean Coffee Austin</a>, a biweekly open meeting</li>
Austin CTO Club</a></li>
</ul>
Everyone who attends these meetings now knows how to run one. And everyone has seen it work effectively in these contexts, even though in the case of Austin CTO Club the group is large for this technique.</p>
People rarely say they already know the technique. It’s been a standard among agile and lean coaches for years. There are many meetings run this way around the world. I’ve seen hundreds if not thousands of people attend them over the years. Why isn’t it more commonly used? It’s not clear to me.</p>
One good context for Lean Coffee in a time-based retrospective meeting. You’ll see all of the topics, whether you get to them or not, and everyone can share their perspective at least at the level of topic creation.</p>
People seem to enjoy Lean Coffee as participants, but that somehow does not translate into trying to lead one themselves.</p>


Dev in a Jail
Unknown — Thu, 21 Nov 2024 17:15:14 -0600
I work a lot on FreeBSD and have made increasing use of jails over time. The bastille</a> and rocinante</a> tools from Christer Edwards make this pretty easy to build and maintain.</p>
These days I do most work in jails, for long-running services like DNS and PostgreSQL, and to isolate programs, like browsers. Most of my dev is in jails as well, where I can wipe the slate clean and start again from fully scripted state at any time.</p>
One of the simple pleasures of developing in a jail is that very little is happening there. Today I had to work on the host as well as in the jail. The volume of processes on the host is large compared with an individual jail, and made diagnosis of one of my bugs difficult. I had lost track of how nice it is to work with a minimal setup that one gets with a focused yet fully capable jail. So this is really a TIR entry – Today I Relearned.</p>


TIL vs TIW
Unknown — Wed, 20 Nov 2024 16:39:16 -0600
An oddity of producing TIL blog entries is that they rarely match the TIL claim. At least it’s that way for me, and I suspect most who try to produce something on a daily basis. It really should be called TIW for “Today I Wrote”.</p>
Even ideas that are easy to write up in short form usually take more than one day to come into full focus for me. And the TIL blog is primarily an exercise in daily writing, so TIW seems more fitting. The mismatch of TIL with the actual production amuses me for no good reason.</p>
I saw numerous random posts under a TIL heading before encountering Simon Willison’s impressive collection</a>. His is a good model to follow.</p>
Today I learned of Josh Branchaud’s TIL repo</a> and its origin story</a>.</p>


Search Path and Extensions
Unknown — Tue, 19 Nov 2024 12:30:56 -0600
Using search_path</code> to implement polymorphism is one of those conveniences in PostgreSQL that can bite hard. Relying on it to select the schema for tables and functions may seem useful, but there is an entire PostgreSQL wiki page</a> dedicated to avoiding harm from malicious use of search_path</code>.</p>
One recommendation from that page is to remove the public</code> schema from search_path</code> altogether. When I tried that I ran afoul of the placement of extensions into the public</code> schema. This is not mentioned on the above page, but does appear in one of its references</a>. Extensions may be “relocatable” to different schemas, but it is not clear to me how common that is.</p>
The more accommodating recommendation is to forbid the creation of objects in public</code> by any non-admin role. That at least avoids the issue of extension functionality disappearing under a restricted search_path</code>.</p>


Man Page in Plain Text
Unknown — Mon, 18 Nov 2024 10:22:43 -0600
Here is a command to print a man page in plain text:</p>
man -P cat page | col -b</span></span></code></pre>
This uses cat</code> as the pager, and strips backspace characters with col</code>. Why would you want to produce plain-text output from man? So you can create embeddings from your man pages, of course!</p>
I am experimenting with this on a FreeBSD system, and today converted 9k+ man pages to text, split them into suitable chunks for embedding, and calculated vectors. The ultimate destination is a PostgreSQL database, where I create a new data table for each set of embeddings and perform RAG queries.</p>
Plain text is gaining prominence as an input format to LLMs. For those of us accustomed to work on the command line this is a very welcome development!</p>


OCR via LLM
Unknown — Sun, 17 Nov 2024 10:00:30 -0600
My stepfather has a box of newspaper clippings from long ago and wants to create digital copies. I asked him to send me a scan. It was an ordinary looking image. Three pairs of text columns, two pairs across the top, one below between the other two, reading in that order. It had some background flecks throughout the image, but was fully legible.</p>
I attached it to a request to Claude with the instruction “Extract as much text as you can from the attached image.” It extracted the full text, including a bullet list in the article, in the correct reading order.</p>
This surprised me. I know little about OCR, but did not expect a general tool to do such a good job. What will this do to businesses built around more traditional approaches to OCR? Do they all need to convert, or at least incorporate LLMs into their operations?</p>
No doubt many more such surprises are coming, and the disruptions they cause will be fun to watch!</p>


A Useful Shell Pattern
Unknown — Fri, 15 Nov 2024 16:52:16 -0600
If you work at the unix command line you probably execute pipelines of programs to test ideas. It’s part of the unix way</a>. Sometimes you need a little more than a simple pipeline. A small shell script here or there is an easy lightweight way to provide a slighly larger pipeline component or to wrap several components for repeated execution.</p>
Often I find myself writing such a script that processes data differently to provide related but distinct manipulations of the data, handle multiple input or output formats, or apply similar processing to distinct source types. When this happens I use the following simple pattern:</p>
shell-script mode [mode-args]</span></span></code></pre>
Here mode selects the shell script execution path, and different modes may take different arguments.</p>
At the top level of such a script I use a case statement that dispatches execution to a shell function named for the mode:</p>
case "$1" in</span></span>
  mode1|mode2|mode3)</span></span>
    "do_$@"</span></span>
    ;;</span></span>
</span>
  *)</span></span>
    printf 'usage statement'</span></span>
    exit 1</span></span>
    ;;</span></span>
esac</span></span></code></pre>
The purpose of the case statement is merely to constrain the available modes. Strictly speaking you don’t need it, though I recommend it.</p>
All you need is to define a “mode function” called do_modename</code> and add the name of the mode to your top-level dispatch. A separate function for each mode mode makes the code flow easy to follow. Different modes can make use of the same utility functions in the script if appropriate. And you can repeat the "do_$@"</code> pattern internally if additional layers of dispatch are needed.</p>
This top-level dispatch to mode-specific functions makes it easy to create clean scripts to handle multiple modes. Adding new modes is highly modular. Modes need not share arguments, as all arguments beyond the mode are handled by the mode function itself.</p>
One mode may even pipe several others together by calling their mode functions, handling the combined arguments at the top level and passing them down as needed.</p>
This is one way in which shell remains underrated.</p>


Split for Embedding
Unknown — Thu, 14 Nov 2024 20:02:39 -0600
Different embedding models have different limits on input tokens. When you want to create embeddings for a large corpus, one major annoyance is splitting the content so that it fits within this limit.</p>
My recent work has centered around Simon Willison’s excellent llm</code></a> tool. He also wrote ttok</code></a>, a command-line tool for counting tokens based on tiktoken</code></a>. While it lacks options to split, it was not difficult to add one in the form of a --chunksize int</code> option.</p>
When invoked with --chunksize</code>, ttok</code> prints output in sets of the given size. This doesn’t make much sense when printing the text out as happens with --truncate</code>. It’s more sensible but not very useful with --tokens</code>, which prints the tokens in legible form. But with --encode</code> it means you get one line of token numbers per chunk. That lets you do this:</p>
ttok -m model -i infile --encode --chunksize 1000 | split -l 1 outprefix.</span></span></code></pre>
Iterating on the chunked output with ttok --decode</code> yields your chunks. Very simple.</p>
After implementing this I discovered a much earlier conversation</a> that covers splitting and converting back to text in one pass. As it happens, I started with that in mind as well, but saw that working with encoded output gave line-oriented results. This meant ttok</code> could do less, and split</code> could do the rest, which appealed to my taste for unix-style simplicity. You can find ttok</code> with --chunksize</code> on the “chunksize” branch here</a>. The packaging is incomplete. If it is useful to you, let me know and I’ll complete the work needed for a proper pull request.</p>


Checking Make Vars
Unknown — Wed, 13 Nov 2024 22:07:50 -0600
If your make target recipes use make variables you may want to test that they are set before using them. This would look something like this:</p>
test -n '$(varname)' || { echo 'variable not set: varname'; exit 1; }</span></span></code></pre>
Rather than sprinkling that sort of code all over your Makefile, you may want to use a dependency instead. If you want to be sure that the variable x</code> is defined, use the dependency needvar.x</code>.</p>
foo: needvar.x</span></span>
        echo "This is foo, and x=$(x)"</span></span></code></pre>
This should fail if x</code> is unset or empty (my common use case). So how is the needvar.x</code> defined? Well that depends.</p>
If you run GNU Make, then it probably looks like this:</p>
needvar.%: .PHONY</span></span></code></pre>
Which works for any variable.</p>
In BSD Make like this:</p>
needvar.x: .PHONY</span></span></code></pre>
with a loop to define the target for each variable of interest.</p>
The package make-needvar</a> implements this so that you don’t have to. It uses the tricks from make-help</a> to handle both flavors of Make. All you need is to include the top-level file, and for BSD Make set needvars</code> to the list of variables you will check. After that, the needvar.x</code> targets are defined, and you can use them in any target that requires a variable setting.</p>
That’s it. Stupid simple, yet effective!</p>


Sqitch Templates
Unknown — Tue, 12 Nov 2024 17:46:40 -0600
Sqitch has a template</a> mechanism that follows the same design pattern as its migration files. This makes it easy to manage templates with Sqitch itself.</p>
My project directory now contains a templates/</code> subdirectory. This is just another Sqitch project top_dir</code>, but for templates. The sqitch.conf</code> file in that directory looks like this:</p>
[core]</span></span>
  engine = pg</span></span>
  template_directory =</span></span>
  extension = tmpl</span></span>
</span>
[engine "pg"]</span></span>
  registry = illegal-registry</span></span>
  client = none</span></span>
  template_directory =</span></span></code></pre>
This means new “migrations” carry the .tmpl</code> extension, and deployments are intentionally broken. I create a new template with</p>
sqitch -C templates add new_template_name</span></span></code></pre>
In the project directory, the parent to templates/</code>, use this in sqitch.conf</code>:</p>
[add]</span></span>
  template_directory = templates</span></span></code></pre>
This also works in the normal user-wide template directory ${HOME}/.sqitch/templates</code> or in the project. I do both, borrowing templates into the project, and then developing them there, merging back into the user-wide templates as appropriate.</p>
It’s simple yet effective!</p>


Variables in Sqitch
Unknown — Mon, 11 Nov 2024 09:32:29 -0600
Sqitch supports variables in the config file</a> like this:</p>
[target "foo"]</span></span>
    engine = pg</span></span>
    uri = your-connection-string-here</span></span>
    registry = sqitch_db_foo</span></span>
</span>
[target.foo.variables]</span></span>
    database = foo</span></span></code></pre>
This makes a variable called database</code> available in the deployment script. I use PostgreSQL, and Sqitch uses psql</code> to deploy. So the variables are available as psql</code> variables in the scripts. What can we do with these variables?</p>
One of my projects uses the same Sqitch configuration for multiple database deployments on the same PostgreSQL instance. Assuming that the database exists, the settings in the config above for registry</code> and database</code> are all the scripts need to function correctly across multiple databases.</p>
Setting registry</code> to a database-specific value prevents Sqitch from mixing up deployments belonging to different databases.</p>
Here’s a deploy file that uses the database</code> variable to create database-specific roles:</p>
BEGIN;</span></span>
</span>
\set ON_ERROR_STOP off</span></span>
</span>
\set _admin_role '_admin_' :database</span></span>
\set _admin_user 'admin_' :database</span></span>
</span>
\set _app_role '_app_' :database</span></span>
\set _app_user 'app_' :database</span></span>
</span>
</span>
CREATE ROLE :"_admin_role" WITH NOLOGIN;</span></span>
CREATE ROLE :"_admin_user" WITH LOGIN INHERIT;</span></span>
</span>
CREATE ROLE :"_app_role" WITH NOLOGIN;</span></span>
CREATE ROLE :"_app_user" WITH LOGIN INHERIT;</span></span>
</span>
GRANT :"_admin_role" TO :"_admin_user";</span></span>
GRANT :"_app_role" TO :"_app_user";</span></span>
</span>
\unset _admin_role</span></span>
\unset _admin_user</span></span>
</span>
\unset _app_role</span></span>
\unset _app_user</span></span>
</span>
\set ON_ERROR_STOP on</span></span>
</span>
COMMIT;</span></span></code></pre>
Disabling ON_ERROR_STOP</code> prevents the CREATE ROLE</code> calls from halting the process if the role already exists. Using Sqitch variables makes reusing deploy scripts across databases surprisingly easy!</p>


PostgreSQL Types by Reference
Unknown — Sun, 10 Nov 2024 20:31:55 -0600
When writing a function to work with a specific table structure, you may have to match the argument types with table columns. If a table column changes its name, you may or may not need to rework the function to apply a similar change. If the table column changes its type, you almost certainly need to change the function to follow the update.</p>
Today I encountered that problem and discovered the option of assigning types by reference. The CREATE FUNCTION</code> documentation</a> says this:</p>

The type of a column is referenced by writing table_name.column_name%TYPE</code>. Using this feature can sometimes help make a function independent of changes to the definition of a table.</p>
</blockquote>
Per the docs, this applies to function arguments and function return values. It also applies to function variables. Maybe it applies elsewhere also, but this alone is a considerable benefit!</p>


Exit Codes in psql
Unknown — Sat, 09 Nov 2024 11:55:25 -0600
In PostgreSQL certain commands are forbidden in transactions or functions, and lack IF NOT EXISTS</code> variants. Examples include CREATE DATABASE</code> and CREATE ROLE</code>. How can one create a database idempotently?</p>
The psql</code> documentation says this about exit codes:</p>

psql</code> returns 0 to the shell if it finished normally, 1 if a fatal error of its own occurs (e.g., out of memory, file not found), 2 if the connection to the server went bad and the session was not interactive, and 3 if an error occurred in a script and the variable ON_ERROR_STOP</code> was set.</p>
</blockquote>
Based on that description, I tried this:</p>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>c</span> '</span>CREATE DATABASE fubar;</span>'</span> -</span>v</span> ON_ERROR_STOP=off</span></span></code></pre>
But this exits 1 if the database already exists!</p>
After this surprise I tried numerous variations on the theme. These versions use -c command</code>:</p>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>c</span> "</span>CREATE DATABASE fubar;</span>"</span> -</span>v</span> ON_ERROR_STOP=off</span>;</span> echo</span> $</span>?</span></span>
ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
1</span></span>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>c</span> "</span>CREATE DATABASE fubar;</span>"</span> -</span>v</span> ON_ERROR_STOP=off</span>;</span> echo</span> $</span>?</span></span>
ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
1</span></span>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>c</span> "</span>CREATE DATABASE fubar; select 'after';</span>"</span> -</span>v</span> ON_ERROR_STOP=off</span>;</span> echo</span> $</span>?</span></span>
ERROR:</span>  CREATE</span> DATABASE</span> cannot</span> run</span> inside</span> a</span> transaction</span> block</span></span>
1</span></span>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>c</span> "</span>CREATE DATABASE fubar;</span>"</span> -</span>c</span> "</span>select 'after';</span>"</span> -</span>v</span> ON_ERROR_STOP=off</span>;</span> echo</span> $</span>?</span></span>
ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
after</span></span>
0</span></span></code></pre>
These versions read commands from standard input:</p>
$</span> echo</span> "</span>CREATE DATABASE fubar;</span>"</span> |</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>v</span> ON_ERROR_STOP=off</span>;</span> echo</span> $</span>?</span></span>
ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
0</span></span>
$</span> echo</span> "</span>CREATE DATABASE fubar; select 'after';</span>"</span> |</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>v</span> ON_ERROR_STOP=off</span>;</span> echo</span> $</span>?</span></span>
ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
after</span></span>
0</span></span>
$</span> echo</span> "</span>CREATE DATABASE fubar;</span>"</span> |</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>v</span> ON_ERROR_STOP=on</span>;</span> echo</span> $</span>?</span></span>
ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
3</span></span>
$</span> echo</span> "</span>CREATE DATABASE fubar; select 'after';</span>"</span> |</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>v</span> ON_ERROR_STOP=on</span>;</span> echo</span> $</span>?</span></span>
ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
3</span></span></code></pre>
These versions read commands from a file that contains only CREATE DATABASE fubar;</code>:</p>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>f</span> test0.sql</span> -</span>v</span> ON_ERROR_STOP=off</span>;</span> echo</span> $</span>?</span></span>
psql:test0.sql:1:</span> ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
0</span></span>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>f</span> test0.sql</span> -</span>v</span> ON_ERROR_STOP=on</span>;</span> echo</span> $</span>?</span></span>
psql:test0.sql:1:</span> ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
3</span></span>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>f</span> test1.sql</span> -</span>v</span> ON_ERROR_STOP=off</span>;</span> echo</span> $</span>?</span></span>
psql:test1.sql:1:</span> ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
after</span> in</span> file</span></span>
0</span></span>
$</span> psql</span> postgresql://postgres@localhost:5432/postgres</span> -</span>A</span> -</span>t</span> -</span>f</span> test1.sql</span> -</span>v</span> ON_ERROR_STOP=on</span>;</span> echo</span> $</span>?</span></span>
psql:test1.sql:1:</span> ERROR:</span>  database</span> "</span>fubar</span>"</span> already</span> exists</span></span>
3</span></span></code></pre>
The presence of additional commands in the file appears to make no difference.</p>
In case it makes any difference, my database connection runs through a pgbouncer</code> instance. I believe this has no impact on psql</code> exit codes.</p>
There are ways to accomplish the goal. I imagine that the under-the-hood logic is clearer. From the outside, this degree of inconsistency seems impressive.</p>


Second Brain
Unknown — Fri, 08 Nov 2024 17:01:34 -0600
Someone recently told me about Building a Second Brain</em></a>. The book presents a “CODE” approach: Capture, Organize, Distill, Express. It reads like an update to Getting Things Done</em></a> with an emphasis on more digital tooling.</p>
But the available tools are improving fast! The development of LLMs, and AI tools more broadly, creates new ways to address each stage of the CODE approach. I want to see how much there is to gain in applying them in this realm. It seems promising.</p>


Separating Storage From llm Embeddings
Unknown — Thu, 07 Nov 2024 15:33:34 -0600
The llm</code></a> program uses sqlite to store embeddings. I want to store my embeddings in PostgreSQL. Rather than try to alter the built-in storage, I opted for the more unix-friendly approach of simply printing the results to standard output.</p>
This was fairly easy to implement based on code stolen from llm/cli.py</code> and llm/embeddings.py</code>. It also leaves in place the logging code, which still uses sqlite.</p>
The llm-vector</a> project implements this. As of this writing it lacks tests, because I haven’t yet figured out how to adapt the tests from llm embed</code> and llm embed-multi</code>. The two top-level functions are llm vector</code> and llm vector-multi</code>. These closely resemble the corresponding embed</code> calls. See the module or its help for details.</p>
A couple of differences are worth calling out here. There is no use of an embedding database, so the model becomes a mandatory argument. The vector</code> versions do not filter out entries based on an input hash. There is no accommodation for metadata or collections. Under the hood it’s somewhat simpler than the original.</p>
What can one do with this new code?</p>
Print the embedding for a single string:</p>
llm vector my-favorite-model -c "The llm-vector module prints embeddings of one or more inputs to standard output."</span></span></code></pre>
Write multiple embeddings to a file:</p>
llm vector-multi my-favorite-model --files files-dir '**/*' > output</span></span></code></pre>
Load embeddings directly into a PostgreSQL table:</p>
llm vector-multi 3-small --files files-dir '**/*' --oformat tsv | psql connection-string -c 'COPY mytable(key,vector,content) from stdin with (FORMAT text)'</span></span></code></pre>
I already use PostgreSQL for my other database work, and now I can integrate llm into my workflow. Once again, unix-style programming for the win!</p>


Programming Throwdown
Unknown — Wed, 06 Nov 2024 07:28:40 -0500
Jason Gauci and Patrick Wheeler have an excellent podcast called Programming Throwdown</a>. I know Jason from the Austin CTO Club</a>. The first episode</a> I listened to taught me about something very useful that I had been looking for, namely GraphRAG models. Check it out!</p>


A gh Alias For json Data
Unknown — Tue, 05 Nov 2024 23:14:22 -0600
The GitHub command-line tool gh</code> has a number of commands that can return json data. For example, this command returns the two fields of data for each repo:</p>
gh</span> repo</span> list</span> -</span>-json</span> field1,field2</span></span></code></pre>
But there is no way to fetch all fields without listing them explicitly.</p>
For commands that take the --json</code> option, using it without any fields, or with an incorrect field, throws an error with a message containing the list of valid fields. We can use that to construct the list of all fields in an alias in the config.yml file for gh</code>:</p>
    j</span>son</span>:</span> |</span></span>
        !gh --json "$(gh "$0" "$@" --json 2>&1 | sed -E '/^ +/!d;s/^ +//;H;$!d;g;s/^\n//;s/\n/,/g')" "$0" "$@"</span></span></code></pre>
With this alias in place we can use gh json repo list</code> to fetch all fields for each repo. While it is not thoroughly tested, I believe this alias works for any command taking the --json</code> option.</p>


Canary Islands
Unknown — Mon, 04 Nov 2024 16:34:33 -0600
Today I spoke with a Ukranian entrepreneur now living in the Canary Islands. I didn’t know the population of the islands exceeds 2 million! He told me a number of other things about the islands that were news to me.</p>

They experience Calima, sandstorms that blow in from the Sahara, and these can last a week.</li>
Deposits from countless years of these storms formed large dunes of light sand, in contrast to the darker volcanic sand and stone of the island itself.</li>
Flights to the Spanish mainland cost as little as 5-10 Euro, and there are many per day.</li>
It is a good environment for children.</li>
</ul>


An llm Plugin Hack
Unknown — Sun, 03 Nov 2024 14:39:11 -0600
The llm</a> package from Simon Willison has a plugin architecture. So I decided to try it. Any reasonable person would begin with a new model plugin. I decided instead to create llm rag</code>, an encapsulation of rag queries that otherwise requires a pipeline of invocations of both llm</code> and files-to-prompt</code>.</p>
The model for this plugin is approximately this:</p>
llm</span> similar</span> "</span>PROMPT</span>"</span> \</span></span>
  |</span> jq</span> -</span>r</span> '</span>[ "<document>", .content, "</document>"] | join("\n")</span>'</span> \</span></span>
  |</span> {</span> printf</span> '</span><documents>\n</span>'</span> &&</span> cat</span> -</span> &&</span> printf</span> '</span></documents>\n%s</span>'</span>;</span> }</span> \</span></span>
  |</span> llm</span> -</span>m</span> "</span>$(</span>model</span>)</span>"</span> "</span>PROMPT</span>"</span></span></code></pre>
This works for its narrow purpose, but is rather inflexible with respect to the many options that llm</code> offers. How do we enable use of those options in this context?</p>
Creating this plugin makes one thing clear. This problem maps very poorly onto the exiting llm</code> implementation architecture. So the result is very messy. The rough idea is to pull in llm</code> code to parse arguments, then run a similarity prompt, package the outputs, and then run the same prompt with the similarity results as context.</p>
I did this two ways, and the results are here</a>. The first is on the master</code> branch. It processes the requests with code adapted from llm</code> and files-to-prompt</code></a>. An unfortunately large volume of code is required for this, as it pulls from multiple levels within the llm</code> implementation. Some options from the original commands are not supported, including templates and conversations.</p>
A second approach is on the subprocess</code> branch. This one focuses on parsing and filtering options, and then runs llm</code> in a subprocess for each stage of processing. This is slightly less messy, and handles all options from each stage, modulo some minor disambiguation.</p>
It still borrows far too much code from llm</code>, mostly to deal with the variety of prompt constructions, in particular templates. Adding a template option to llm similar</code> would clean this up some.</p>
Most of the remaining mess pertains to the crude reconstruction of the command-line arguments for the subprocess. Perhaps using --</code> separators between options sections is a better approach. Maybe there is a feature of click</a> that I overlooked. Even if so, the code does show some simple ways to work with the structures from click.
If you are interested in creating your own llm</code> plugin, this is probably a good anti-pattern example.</p>


Convert Between json and yaml With yq
Unknown — Sat, 02 Nov 2024 08:20:25 -0500
One of my experimental projects involves fetching data from GitHub with gh</code>, editing the output, and then posting the results back to GitHub. It’s easy to produce json output from gh</code>, but who wants to edit in json? I’d rather use yaml. Enter the yq</code></a> tool.</p>
It works much like jq</code></a>, but also handles conversions from json to yaml and vice versa:</p>
echo '{"here":"there"}' | yq -pj -oy</span></span>
echo 'here: there' | yq -py -oj</span></span></code></pre>
So now I can convert to yaml, edit there, and convert back to json before posting t the API.</p>
This is my only use of yq</code> so far. I use jq</code> elsewhere, and see yq</code> as useful mainly for this type of conversion. One more problem solved!</p>


Willison's llm
Unknown — Fri, 01 Nov 2024 08:20:49 -0500
Simon Willison has created a marvelous tool for experimenting with LLMs at the command line: llm</code></a>. It employs a plugin architecture and offers the ability to connect to hundreds of models, both remote and local, and supports direct prompts and embeddings. This tool makes it extremely easy to experiment with different LLMs.</p>
It uses sqlite as the back-end data data store. While this is admirably simple, it is also somewhat limiting. I almost immediately produced embedding sets too large for sqlite to handle well. I hope to extend llm</code> to support a PostgreSQL storage layer. This part of the program is not architected for extension by plugin, so the update is not so straightforward.</p>
The llm</code> program offers many features:</p>

Extension through plugins</li>
API access to underlying features</li>
Model aliases</li>
Conversations threading together multiple requests</li>
System prompts</li>
Image attachments</li>
Prompt templates with variable substitution</li>
Request logging</li>
Extensive documentation</li>
</ul>
There are also useful related tools:</p>

files-to-prompt</code></a>:
Structure files for prompt inclusion</li>
ttok</code></a>: Count prompt tokens</li>
</ul>
If you want to experiment with LLMs and RAG models, I don’t know a better tool to learn with. Try it out!</p>


Make Help
Unknown — Thu, 31 Oct 2024 08:34:01 -0500
If you use make</code> in your projects, providing simple help for users is a common problem. What targets exist? What do they do? If you use nested Makefiles, where are the targets defined?</p>
The make-help</a> project provides a very simple way to add documentation to your Makefiles that addresses all of these questions. It uses comments in the Makefiles themselves to provide the information. It handles nested Makefiles, and works with both BSD and GNU make</code>.</p>
Add three makefile fragments to your project, and add a single include</code> statement to each Makefile that you want to document. Place that statement above all other targets in the Makefile to make it the default. Then add comments to the Makefile. That’s all there is to it!</p>
The project is in the public domain, so have at it! Other users of your Makefiles will thank you later.</p>


Jump to Next Row
Unknown — Wed, 30 Oct 2024 07:28:40 -0500
A handful of records go into one of my Google Sheets on a daily basis. It’s most convenient to add them at the bottom, but scrolling there is tedious. The hyperlink()</code> function offers a way to avoid the tedium.</p>
A formula in the header lets me jump to the first row where I can add new elements. This works by calculating the row number and then calling hyperlink()</code> to jump there. After jumping there you must select the cell before editing. There’s no apparent way to select for editing with hyperlink()</code>. Also, the chosen cell is aligned at the top of the view, so you do need to scroll if you want context. Nevertheless, this does eliminate scrolling, the worst part of finding the right row.</p>
Here is a formula that locates the first row with an empty cell in column B</code>, and then selects column A</code> in that row. The header cell that contains the link reads “Next Row”.</p>
=hyperlink("#gid=SHEET_ID_HERE&range=A" & MATCH(TRUE,ARRAYFORMULA(LEN(TRIM(B:B))=0),0),"Next Row")</span></span></code></pre>
That’s it!</p>


Replacing pip with uv
Unknown — Tue, 29 Oct 2024 08:16:31 -0500
Recently I discovered uv</code></a> as a potential replacement for pip</code>. A single use paid off the switching costs.</p>
It has three main advantages over pip</code>:</p>

Speed</li>
Correctness</li>
Independence from the project</li>
</ul>
The speed advantage is obvious. The other two advantages warrant explanation.</p>
I had installed Simon Willison’s excellent llm</code></a> CLI tool for LLM use. When I used pip freeze</code> to capture the current installation, it produced an invalid version for an sqlite3</code> library, necessitating a manual fix.</p>
Some of my work uses FreeBSD. After installing the llm</code> plugin llm-gpt4all</code>, my project broke. Installation ran without error, but the plugin threw runtime error saying that FreeBSD is not supported. Failing to detect the lack of support at install time and relying on a run-time error is bad behavior to discuss elsewhere. At least the fix is easy, just uninstall llm-gpt4all</code>.</p>
Not so fast! When attempting uninstall, pip</code> throws the same error! So pip</code> is hosed because the project is hosed.</p>
I created a new setup with uv</code> instead of pip</code> and found that uv</code> freezes correct version numbers and also successfully uninstalls the module throwing the run-time error. That’s enough benefit to justify the switch.</p>


Welcome
Unknown — Sun, 12 Jun 2022 18:31:58 -0500
“Alright, move on. Nothing to see here. Please disperse.”</p>
– Frank Drebin</p>
SuperScript

Back It Off a Quarter Turn

Sorting Networks

A Judgement Amplifier

Making the Cut

Prototype vs Implementation

Other Approaches?</h2> There are doubtless many more ways to address the question with some combination of constraint, education, and planning. What approach would you take?</p>

Document-Driven Development

Defining Hypotheses

MVPs Reveal Preference

Cheap and Cheerful

Letters in LaTeX

Google Sheet Sort

Hot-Potato Package Delivery

Idempotent Home Directory Setup

The Dismal Science

A Two Card Strategy

Interviewing With AI

Scripting asciinema

Pipelines Without Pipes

Friedman, Pigou, Coase

How Fast Is Gas?

The More You Owe the More You're Worth

2 Second Lean

My Best Losing Bet

AI and Individual Contributors

A Bike Shifting Heuristic

In Search of Ignorance

Build to Buy

A Pianist's Scales

Aider First Impressions

Training Data Shortages

Austin CTO Club

What’s In A Name?</h2> A brainstorming session with some like-minded friends raised a number of basic questions:</p> What format makes sense?</li> Where should we host it?</li> Who will sponsor?</li> Who may join?</li> How do people register?</li>

Complex Judgement</h2> There are explicit attendance criteria:</p> Senior technical role (exact title may depend on org size)</li> Manager of a team</li> Experience in hiring and firing</li>

Succinct Themes Work</h3> Paragraph-length theme suggestions are not uncommon. Taking the time to distill them into a single punchy phrase helps catch people’s interest.</p>

Introductions After Topics</h3> We usually run 25-30% new attendees and the rest returning. A quick round of introductions helps people connect. Brainstorming topics before introductions lowers inhibitions in topic creation, as compared with the reverse order.</p>

Everyone Loves A Givaway</h3> The meeting-branded Yeti mug we raffle off at the end of the meeting is very popular.</p>

Finding Us</h2> Interested in learning more about the Austin CTO Club? Find us here:</p>

Necessity and Luxury

Which Way Will Models Go

Spontaneous Order, Lunchtime Edition

MCP and Firecracker

Does Art Make the Cut

Leavings of Many Deaths

Summary and Audience

Listening Is Underrated

Step CLI

Training Intuition

Evolutionary Reordering

Scaling and Standardization

Moats and AI Revisited

Monopoly: Exclusion</h2> A patent is one common form of monopoly. Utilities with exclusive right to serve a region is another. These grants of monopoly are defensible insofar as the grantor allows it. They let you exclude competition.</p>

Linuxulator for the Win

Fragments for llm

Dual OS Single Boot

Solo Unicorn

What Cannot Be Replaced?</h2> What about fundraising? Bootstrapping aside, a startup founder must handle this. An investment in a company of one founder is an investment in a team of one. I don’t see how that can be outsourced.</p>

A Zero Founder Unicorn?</h2> At what point is the entrepreneur outsourced alongside everything else? Will this only occur with a bootstrapped business, or will a VC invest in a team of zero?</p>

Upside Potential

The Data Moat

Multiple Llamafiles With llm

More Moats and AI

Python CVS Woes

Moats and AI

What Did I Miss?</h2> Should something else make the cut? Am I off the mark with the three that I’ve chosen? What say you?</p>

Richard Epstein at Civitas

Meetup Attendees

Wrapping jq select

Jujutsu Improves on Git

Editing PDFs With qpdf

The New Programming

Who Owns Your Dot Files

Bash the Cache

Debian JRE Package Bug

Linux Jails With VNET

mawk Is Not awk

Cognitive Load

fzf Plus rg

Locale for Sorting

Other Approaches?</h2>
There are doubtless many more ways to address the question with some combination of constraint, education, and planning. What approach would you take?</p>

Succinct Themes Work</h3>
Paragraph-length theme suggestions are not uncommon. Taking the time to distill them into a single punchy phrase helps catch people’s interest.</p>

Introductions After Topics</h3>
We usually run 25-30% new attendees and the rest returning. A quick round of introductions helps people connect. Brainstorming topics before introductions lowers inhibitions in topic creation, as compared with the reverse order.</p>

Everyone Loves A Givaway</h3>
The meeting-branded Yeti mug we raffle off at the end of the meeting is very popular.</p>

Finding Us</h2>
Interested in learning more about the Austin CTO Club? Find us here:</p>

Website</a></li>
Meetup</a></li> </ul>

Monopoly: Exclusion</h2>
A patent is one common form of monopoly. Utilities with exclusive right to serve a region is another. These grants of monopoly are defensible insofar as the grantor allows it. They let you exclude competition.</p>

What Cannot Be Replaced?</h2>
What about fundraising? Bootstrapping aside, a startup founder must handle this. An investment in a company of one founder is an investment in a team of one. I don’t see how that can be outsourced.</p>

A Zero Founder Unicorn?</h2>
At what point is the entrepreneur outsourced alongside everything else? Will this only occur with a bootstrapped business, or will a VC invest in a team of zero?</p>

What Did I Miss?</h2>
Should something else make the cut? Am I off the mark with the three that I’ve chosen? What say you?</p>
