I Taught a Robot to Read the News So I Don’t Have To

Newsletters are dead! AI is now the first and last word in the daily news. If you are a journalist or a content creator, it is time to find a new line of work.

A robot reading the newspaper, digital art.png

By now, you have probably heard a declaration like this in the context of every field and every job. Maybe one day, AI optimists will be correct, and computers will do everything for us. We aren't there yet but we are getting closer.

I have spent the last few months working to build a newsletter compiled and written entirely by AI. This newsletter grew out of my frustrations with product management news. Product is a field that is rich with long-form and strategic content. But searching for daily product management news doesn't return much. And I wanted to consume this content, not create it. So I decided to see if AI, specifically LLMs and GPT could live up to their hype. Could they serve me daily news, that was relevant to product managers, and explain to me why I should care? So far the answer is maybe. It is not perfect, but it does as well as I could.

A lot went into creating the newsletter, if you are curious to learn more, you can check out the README on GitHub. But I did want to give some details on how I used GPT. I took some conventional and non-conventional approaches. LLMs are often misused but when properly applied, they can do some incredible things. So let's look at some of the ways I used GPT in the newsletter.

Getting started

For everything I am going to describe here, I used either the gpt-3.5-turbo or gpt-3.5-turbo-16k APIs. The non-16k model was desirable because it was the best mix of performance and cost. For portions where I needed to process a lot of content, the 16k model can handle more tokens, as implied by its name.

I first started testing the concept for this newsletter from my phone, while on vacation. That meant I didn't have a lot of screen real estate or great tooling. But I was able to test the basic premise. I passed GPT the text of an article and asked it to tell me: if it was relevant to product management, and if it was, to provide me with a summary of the article. This worked better than I expected. But I learned at scale this approach was not reliable. Asking GPT to do too many things at once leads to some weird results. So I broke that one prompt up into a few. And then as the newsletter got more complex I added a few more along the way. Let's dive into each of those prompts.

Screen for Relevance

CleanShot Sublime Text-2023-08-15.png

First off there are some topics that I do not want to include in the newsletter. The topics are all based on personal preference, they may be relevant to other product managers but didn't make sense for my needs. The nice thing about using a prompt is that you can turn topics on and off by updating the prompt. Someone could take these prompts and adjust them to a different domain or area of interest and have a newsletter personalized for them.

CleanShot Sublime Text-2023-08-15-2.png

Then I asked it to review individual articles and determine if they are relevant for product managers. This prompt also asked it to summarize the relevant ones but I moved that to another part of the flow. As I mentioned, asking it to do too many things at once was leading to weird outputs.

Key learnings:

Summarization and Intro Generation

CleanShot Sublime Text-2023-08-15-3.png

Once articles have been vetted for relevance we send the text to GPT and ask it to create a summary. There is a limit to the amount of text that can be sent and many articles surpass the limit. The options were either to use the more expensive 16k model or limit the amount of text per article. Doing some testing I found that sending partial text still led to quality summaries. Each summary details what the article is about and explains why it is relevant to product managers.

CleanShot Sublime Text-2023-08-15-4.png

Once all the summaries are generated they are all passed to GPT with a unique theme for the day. GPT uses the theme and the stories for the day to generate a unique intro for that day's newsletter.

Key learnings:

Deduping and Domain Authority

CleanShot Sublime Text-2023-08-15-5.png

In the initial scrape of articles, more than one article will likely cover the same story. A recent example was the launch of Threads. There were at least 10 articles on this topic pulled into the newsletter flow. When this happens I do not want to include them all in the final output, I only want to include one. This leads to a need for two things to happen.

First, I need to identify the duplicate stories. To do this, I send all the URLs for stories to GPT in one payload and ask it to dedupe based on the keywords in the URLs. I found a way to do this through scikit-learn which is likely a more technically sound approach. But the whole point of this was to test the limits of GPT. I also tested sending the text of the articles with the URLs to help in deduping. But I found that GPT did almost as well with just the URLs, so I pulled the text out to keep things simple. Once it has a list of deduped stories it returns the URLs to include in the newsletter.

The second thing to think about with duplicates is deciding which source to use for a story. I looked at using a Domain Authority tool like Moz or Ahrefs, but they were expensive. Instead, I asked GPT if it knew how tools like Moz generate their domain authority scores. It explained to me how they do so and generated some sample scores. I updated the prompt to include instructions to use domain authority when selecting a source.

Key learnings:

CleanShot Freeform-2023-08-15.png

All these pieces get combined into a flow that looks like the above. The result is an email that looks like this recent version of the newsletter.

CleanShot Safari-2023-08-16.png

This is a great time to be building things. The speed and availability of technology make it possible for everyone to be a builder. Incredibly powerful tools are available to anyone willing to put in the effort. They let someone like me, generate content each day with no effort and almost no cost (less than $1 a month). And while I did this for product management, it could be done for any topic. It would just take a few changes to the sources and prompts. So take a chance on AI and have it build a newsletter for you. If you want to learn more about how I built the entire flow and build your newsletter check out GitHub.