AI Rewrites Technical Work

There is a revolution underway. A piece of technology is set to transform society. It promises to unlock things like universal knowledge and social equality. But we're not stopping there, it is also going to transform the way we work and unleash a new generation of creators. It's obvious what this technology is.

It's the typewriter coming to the market in the 1870s. It promised to make a huge impact, including making people healthier and changing the economy. It would sweep through offices overnight and change how business was done. Even America's favorite author, Mark Twain, couldn't resist using one. While some of this did come to bear, it was not immediate, it took decades for full adoption. And even then many of the promises were nothing more than hype.

The recent AI wave promises many of these same things. It will also take decades to see which ones play out. But, like the typewriter, there are some practical applications for AI. One promising area is in how we work with data and code.

AI is making it easier to understand and query your data. It can be your guide in understanding codebases and debugging code. And it is great at explaining these concepts to others on your behalf.

The good news is you don't need to be an AI specialist to harness this power. All you need is a few tools and some creativity. With a little effort, you can improve the way you work. AI simplifies complex tasks, uncovers insights you may have missed, and saves time. Let's dig in, after all as Twain said, "The secret of getting ahead is getting started."

SQL

SQL is one of the most common technical languages in the world. Most of us likely use it daily. Since it is a language it makes sense that LLMs can do a lot to help us here.

Have you ever started a new job and been handed a set of SQL queries or reports? You are told they are important, but you have no clue how they work or what they are for. This has happened to me several times. Usually, I would spend a bunch of time figuring out what the queries do and how the metrics are defined. But not anymore. With a quick script and API call, AI can do it for us. Give it a complicated query and it can explain what it does in an easy-to-understand way. Not only that, it can also optimize the query and rewrite it to perform better. CleanShot Freeform-2024-02-04.png

Maybe you want to take this one step forward and understand what the queries and data look like. It's an easy task for AI to take a query and turn it into an elegant visual flow. CleanShot Safari-2024-08-30.png

Databases

SQL is the language of databases, so it makes sense that LLMs can do much more beyond explaining queries.

Using a combination of Python, SQL, and LLMs you can have an AI agent inspect an entire database for you. It can describe the the data in each table, and give you suggestions for how to use that data. Even more helpful, it can also tell you how the tables relate to each other and how you might use the joined data. Now you can onboard a new database in a few minutes and become an expert within a few days. CleanShot Safari-2024-02-04-2.png

Maybe you're a visual learner. With one quick prompt, you can have this same agent generate an ERD for you. CleanShot Safari-2024-08-31.png

In the real world, your data is much more complicated than these simple examples. You likely have many sources of data in the same database. How do you make the most of that data? Combining it will likely yield unique insights. But what if you are unsure of how to join the data? Let AI do it for you. In this example, AI explains how we can join HubSpot data with Google Analytics data. It not only gives us the technical details needed to make this happen, but it also provides suggestions for using the combined data. 

Database Analysis and Join Recommendations

Key Join Points
1. Email Address: 
   - Hubspot's `contacts.email` can be matched with Google Analytics' `sessions.client_id` (assuming email is used for client ID).

2. Website Domain:
   - Hubspot's `companies.domain` can be related to the domain in Google Analytics' `pageviews.page_path`.

3. Timestamps:
   - Various timestamp fields in both databases can be used for temporal analysis.

Joining Strategies
1. Contact-Session Matching:
   - Join Hubspot's `contacts` table with Google Analytics' `sessions` table using email/client_id.
   - This links individual contacts to their web sessions.

2. Company-Pageview Analysis:
   - Join Hubspot's `companies` table with Google Analytics' `pageviews` table using the domain.
   - This connects company data with their website interaction data.

3. Deal-Conversion Correlation:
   - Join Hubspot's `deals` table with Google Analytics' `conversions` table using timestamps and potentially contact information.
   - This can reveal how web activity relates to deal progression.

4. Activity-Event Mapping:
   - Join Hubspot's `activities` table with Google Analytics' `events` table using timestamps and contact/session information.
   - This can show how CRM activities correlate with website events.

Recommendations for Using Joined Data
1. Customer Journey Analysis:
   - Map a contact's website interactions (from GA) to their CRM lifecycle stages (from Hubspot).
   - Identify which web behaviors are most associated with progressing through the sales funnel.

2. Content Effectiveness for Deal Closure:
   - Analyze which pages (from GA pageviews) are most frequently visited by contacts associated with closed deals (from Hubspot).
   - Use this to optimize content strategy for better conversion rates.

3. Campaign Performance Evaluation:
   - Correlate marketing campaign data (from GA) with deal values and closure rates (from Hubspot).
   - Determine ROI for different marketing initiatives.

4. Personalized Engagement Strategies:
   - Use web behavior data (from GA) to inform personalized outreach activities (recorded in Hubspot).
   - Tailor content and communication based on observed online interests.

5. Predictive Lead Scoring:
   - Combine web engagement metrics (from GA) with CRM data (from Hubspot) to create more accurate lead scoring models.
   - Identify high-value prospects based on both their company profile and web behavior.

6. Customer Segmentation Enhancement:
   - Enrich Hubspot's company and contact segmentation with web behavior data from Google Analytics.
   - Create more nuanced customer segments for targeted marketing and sales approaches.

7. Churn Risk Identification:
   - Analyze changes in web engagement patterns (from GA) alongside CRM activity data (from Hubspot) to identify potential churn risks.
   - Implement proactive retention strategies for at-risk accounts.

8. Product Development Insights:
   - Correlate feature usage or interest (inferred from GA page views and events) with customer characteristics and deal sizes (from Hubspot).
   - Inform product development priorities based on features that attract high-value customers.


By leveraging these joined datasets, organizations can gain a more holistic view of their customers, optimize their marketing and sales processes, and make data-driven decisions across various business functions.

And once again it can show you how it all comes together. CleanShot Safari-2024-08-31-2.png  All that data gives you a lot of power but it also brings risk. There is likely data that you do not want to store in your database or that you do not want others to access. Again AI is our friend here. It can scan your data and surface all the columns that likely contain PII/PHI. The great thing is you don't need to expose the underlying data, AI can do this based on the column headers. And it works well even in cases where fields are oddly named. PHIPII Scanner Results.png 

Code

What if you spend time working on code rather than digging into data? AI can do some great things here too. One of my favorite uses of AI is diagramming how a code base works. By feeding a GitHub repo into an LLM you can quickly learn how all the files and functions in a code base work together. In this example, I gave an AI tool the URL for one of my repositories. The AI inspected all the files and mapped how the code works together. CleanShot Google Chrome-2024-08-31.png  This is a relatively small code base. But as context windows grow, and other techniques emerge, a similar approach could be applied to large codebases. This gives everyone an easy way to get onboarded to a new codebase.

Documentation

Let's say you've already done the data work. And you've written a bunch of code. Now you have one of the coolest products on the market. That's a great place to be, but it's also a pain because everyone wants you to explain it to them. Your team likely has a huge repo for the code and you have a lengthy document explaining how it all works. But those are way too complicated for internal business stakeholders. And they're definitely too complex for customers. So now you need to write up simpler versions of the documentation. Or you can have AI do it for you. Give it the repo and the documentation and with a few prompt templates it can create you as many versions of the docs as you need. Al Explainability Card Generator.png

Back to that typewriter revolution, Mark Twain was also purported to say, "Please do not use my name in any way. Please do not even divulge the fact that I own a machine. I have entirely stopped using the typewriter, for the reason that I never could write a letter with it to anybody without receiving a request by return mail that I would not only describe the machine, but state what progress I had made in the use of it, etc., etc. I don't like to write letters, and so I don't want people to know I own this curiosity-breeding little joker."

I am prone to agree with Mark Twain in most cases but not here. We should be talking about our machines and sharing how they can be useful. Now, it's your turn. How are you using AI in your data work or technical projects? What unexpected ways have you found to integrate AI into your daily tasks? Send me a letter (or better yet an email) with your best ideas.