Nate Sullivan

I've spent a lot of time trying to get Resolution to represent the most complicated scenarios it possibly can. Compared to the average user, I'm (unsurprisingly) pretty good at doing this, even if some of the results are kind of ridiculous.

However, in doing so, I've learned that some of the biggest "aha!" moments in the application aren't with complex models. They are with really, really simple ones, like the one I built today.

Considering User Types

Here's the situation. I'm helping someone figure out yet another go-to-market plan (hopefully better than Resolution's), and one of the things we're budgeting for (unlike Resolution) is some money to acquire users. At first, I just threw all these different people in a big "user" bucket, and assigned a single set of costs and benefits to that big bucket.

However, this turns out to be more than a bit too simple. We have powerful, important, advocate-like users with much higher acquisition costs and network effects than normal users, or especially casual users (like students). I really need to consider what the user breakdown by type is going to be -- not just to anticipate my costs, but to understand my downstream network effects. Unfortunately, I'm working in a spreadsheet, so there are a lot of annoying things about trying to take this stuff into consideration.

Brittle Formulas

Again, I'm in a spreadsheet. For every additional factor I want to consider, I either need to manually code it into my formulas (and have to update it umpteen times if it changes), or find a place I can stash it, and then reference that value. Either way, if I suddenly change the way I'm doing this (by, say, referencing one of several potential values in each place instead of using the same one all the time), I have a lot of refactoring to do, which usually leads to me making mistakes.

Unclear Inputs

The driver of my user acquisition numbers is my user acquisition budget. But of course, we're not totally set on what that should be yet. In fact, as is often the case, the answer is really "as low as possible", but we don't know what that is yet. So I can't just say "the budget is $20k", nor can I say "the budget is whatever it takes to get 100 power users and 400 casuals". I have to give and take on both sides of that to come up with something that makes sense.

Rates Up For Grabs

This is one of those classic Resolution-y scenarios where no answer is ridiculous until you see it, at which point it's obviously ridiculous and you want to make sure you never see it again. What's a reasonable acquisition cost for a power user? A hundred dollars? "That's nothing!" A thousand dollars? "THAT'S INSANE!!!" Okay, okay. I need to be able to take these different costs on and off the table, and probably put some floors and ceilings on them.

What a Flex

Basically, I have what seems like a really simple math problem, except I want a whole bunch of different things to be able to flex -- and of course, I want to be able to go "backwards", where I can use money to determine user outputs, OR I can request certain amounts of certain types of users at certain costs and see that affect the required "input".

Aaaaaand.... here it is.

To be honest, this was even simpler when I started. I simply had my pile of money, my three user groups, and my three costs. All I really wanted to do was figure out what reasonable acquisition costs would be for the goals we have, but once I had that established I immediately wanted to plug in some of the downstream effects, so I just... added them.

Where the model really shines, though, is when the questions come (and they always do):

• Can you make this work with 15k? How does that affect the total user base?
• What if we spent a lot more money on influencers and a lot less on the rank and file users?
• What if referrals are actually stronger from senior staff than from influencers?
• What's the cheapest way to get me to 500 users?
• Follow-up: "Okay, not that way, what else can you do?"

I built this model in 5 minutes. It's not complicated. But I poked at it and tried different scenarios for the next 20, and I've come back to it several times since. Scenarios! Flexibility! It's all there, baby!

As mentioned earlier this summer, our CTO Brian Cunningham has decided to reassign all non-essential personnel towards "short-term revenue generation", which means I have to go back to work to fund this incredible enterprise.

Since I need flexibility to also run this high-tech side-hustle and occasionally hug my children, I'm going the contracting/consulting route.

Building a Proposal

I've done consulting before, but I'm not much of a business development person, and I have a hard time turning what I can do for people into blocks of hours and amounts of money. For one project, I was asked if I could deliver a whole host of different things, when in reality I have (at best) a couple hours each week available to dedicate.

What should I work on? How much work should I put into each thing? How long would the chosen pile of things take if I worked on this for 6 hours a week, or 8?

Hilariously, I started building a spreadsheet to figure this out, but I quickly ran into the quintessential Resolution-brain problem -- I wanted to be able to change both sides of my model, so I could see what how many hours (and thus, weeks) I'd need with different project expectations, OR, crucially, what would happen to my project outputs if I had more or less hours to work with. No spreadsheet could support both of those things in a single model, because that's not how spreadsheets work.

The Resolution Alternative

Once I realized that I was really just allocating hours and turning them into different things (task-allocated hours, or maybe project outputs on one end, and weeks on the other), using Resolution made way more sense. I used the Units of Hours & Weeks for most of the model, and a few project outputs like Assets & Orgs, so I could play with different rates. Those rates measure things like "how many hours to make a one-pager", or "how long does it take to fully assess one competitor". When I couldn't specify project outputs, I just... didn't do it. Instead, I let the hours speak for themselves. Setting up a CRM will take a while, but it's not based on number of contacts or whatever.

I found this whole exercise to be incredibly helpful. In classic Resolution fashion, I felt much better confidently stating that this work would take months, because I had the receipts, and when you look at the individual pieces, they make sense.

Here's the model. Seriously, check it out! You could do this with just about any form of time management.

Back to work. Resolution will be humming along like this for the next few months, so keep on building and drop me/us a line if you have any questions or issues. But we won't be putting out new features for a bit while we save up our developmental ammo. Trust me, no one is more eager to take this thing to the next level -- not just because I invented it, but because I use it for stuff like this!

Unelected! Unpopular! Lifetime appointments! Self-enforced ethics rules! Very little documentation defining its role! The Supreme Court really does have it all, when you stop and think about it.

And "government at its finest"? I think you're on pretty safe ground with "the Supreme Court is important", or my personal stance of "it was a decent catch-all until we started using it too much". But finest?

It's just a tough quote. I definitely would have gone with "the volunteer fire department" or something for "what do you think is government at its finest". If they had specifically asked me about the Supreme Court... hmm... wait, I've got it.

Perfect. No notes.

(Editor's Note: They ruined Twitter, so now when something makes me crazy I have to post here like it's 2007 again, or else text my friends, and I'm giving them a break this time.)

Major news in a sport I don't care that much about, but have been unable to ignore because the lawsuits are too interesting.

This is all fine and probably makes sense, but there are some pretty funny elements to it when you look at how all of this will eventually get sorted out.

Still to be determined is how players like Brooks Koepka and Dustin Johnson, who defected to Saudi-funded LIV Golf for nine-figure bonuses, can rejoin the PGA Tour after this year.

- ESPN

Um... yeah??? So basically these guys bolted (no judgment) for huge piles of money, and the price they paid for those huge piles was that they weren't able to be in the PGA Tour anymore. Well, that's all behind us now, right? Except... they still have the huge piles of money! And the other guys don't!

Nine figures, man. Nine figures.

Anyways, I know who I want smoothing all of this over, and so do you.

We like to stay ahead of the curve here at Resolution. So when I heard people were writing actual web applications with large language model chatbots, I spent $20 on the GPT4 version of OpenAI's ChatGPT and got right to work.

Here's a little window into today's working session:

Is this easier than working with skilled, experienced professionals? In a way... absolutely! This thing does exactly what I tell it, works incredibly fast, and is happy to allow me to change my mind whenever I want, as many times as I want. I feel absolutely no guilt at all doing things like pivoting completely and dumping an entire code base that would cause normal, human software developers to push me into traffic or at least passive-aggressively respond to my meeting invites with "MAYBE". (BTW, I invented that move, so... respect.)

However, there are downsides. For instance... well, here's where we stand as of today, product wise.

Pretty bad! I think the math works, but it's hard to tell because this entire thing is totally illegible, despite the cheery confirmation of my robot engineering department that the problem had been addressed. (UPDATE: The math definitely does not work, at all.) It's hard to say that I can't make this any better, because I probably can! I'll keep yelling at this thing every couple of hours and it will keep making changes without even a scintilla of human empathy or understanding and send them to me in chunks of truncated Python that I copy and paste in a text editor with a hopeful, utterly clueless look on my face.

The future is here! Somehow, though, I don't think the AI Edition of Resolution is going to surpass the slow-cooked, human-architected production version anytime soon. Sorry, Brian.

AI bots make stuff up. This is not new; in fact, it's so commonplace that people aren't even phased by it. Rather than question the value of the bullshit robot, we've started to doubt the existence of non-bullshit.

Unfortunately, this problem isn't going away. And while there's a weird consumer religion out there that machine learning products somehow fix themselves ("sure, it's bad now, but every mistake is just the model getting better!"), that's not going to happen when bots learn from the internet and the internet is increasingly just content from robots that learned how to write content by reading the same internet.

The snake-head is already eating the snake on the opposite side, man. We're here.

Sometimes the product IS the ethics

Generative AI skeptics have been pretty clear that there's an alarming correlation between ethics teams being dissolved and tons of new generative products coming to market. You don't have to be a conspiracy theorist to see that obviously the ethical quandaries of bullshit robots and idea-smushing-together algorithms are significant, and that fully staffed ethics teams were probably putting a pretty high burden on getting stuff out the door.

Well, they're largely gone now, so here come the products. They are very ambitious, and the main ethical effort made on most of them is to say "don't use this for anything" somewhere in the terms of service. That's pretty gross from a personal/moral/ethical standpoint, but in this particular case, it's more than that. The ethical objections to this were partially about poisoning people's minds with plausible fiction generated at scale with little effort. But some of them were also about destroying the content soul of the internet that these very products depend on to function.

We've blown right past that! Chatbot Tech is now a tourist beach resort that dumps its sewage directly into the ocean, and we're barely out of the gate with this stuff.

This could have been good fine

If this was a serious product, to me, you'd reverse the stack here. Output tools are not that interesting, because the output isn't particularly good. At best, it's fine and extremely cheap. But the input -- that's the good stuff. That's how you'd deal with things like misinformation, "hallucinations", and things like that. I don't want tools to generate output that's been trained on a bunch of garbage. If something is going to feign understanding, at least feign the understanding I want and don't start mansplaining to me about things I know you don't know.

But my guess has always been that LLMs need that first "L" to be so large that restricting the dataset to trusted info was always going to be a problem. That you could theoretically do it, and that you can probably do it today with some tools, but that either (a) with a smaller, more restrictive data set you can't actually generate anything as human-ish as what the general purpose chatbots are barfing up and delighting everyone with, or (b) to work at all it would still need to reference other training, and there's no viable way to say "look at this data to learn English, and this data to learn facts", because there are no facts, we're just spitting out tokens here.

Seriously, I'm guessing on this. I'm not a linguist OR an ML engineer (clearly), but I do know how people bring products to market when they want to make money, and one clear sign that something is impossible to do well is when the guy winning the race concedes a necessary portion of his/her market to a "network of partners" or developers and says "you guys figure this out, we're just here to support you". Best case scenario, you get the iOS App Store (which is still hell for a lot of developers, but we did get a lot of great apps along with limitless terrible ones). Worst case, you get Salesforce, where the last mile is a death march you pay a consultant $5,000 an hour to slog through with you and literally no one is happy, ever.

Ethically, owning the model should suck, because the model is what makes everything terrible. Unless, that is, you can convince everyone that (a) the model is a child so it both can't be controlled AND you should feel bad about yelling at it, or (b) it's actually the apps ON the model that need to take responsibility, even if that is impossible, or more likely, just not important to said app-makers.

I think this is where OpenAI and the likes are headed. They can't control this, and that's fine right now because they don't need to. The magical, self-healing lie non-experts like me tell ourselves about ML will buy them plenty of time, and by the time we realize this will always be bad OpenAI will have shifted to "we enable the developers" and "tools are tools, they can be used for good or evil and we think everyone should use them for good" with absolutely no blame-able hand on the steering wheel.

Plus, all the current execs and employees will leave and sit around being rich and blaming 2028's management team for losing the "mission" of the company.

Couple of big announcements here at Resolution HQ. Before handing it off to my new favorite algorithmically generated executives, I'll sum it up for search purposes --

1. There's a brand new onboarding program. Just open the Scratchpad and follow the prompts for a Resolution walkthrough based on a model you can actually relate to.
2. Payments are now working. If you need to save more than two models, and/or you want to see Resolution get better, we've got a solution for you. Here's more on our thoughts on pricing if you're interested.
3. The Model Index has a bunch of good stuff in it, and you can search the entire thing without even opening an account. Just to be absolutely clear, the Model Index is hand-curated by me. Just because you make or share a model, it does not mean it goes in here. I have no idea what you all are creating and only post things that (a) I make, or (b) a Resolution users shares with and says "add this to the index, it's cool!"

That's it. Now, here's AI-generated Resolution CTO Brian Cunningham with another important announcement.

Well said, excu-bots!

In the Wall Street Journal today:

Obviously, I immediately thought of my favorite mayor.

And with that, I yield the floor.