What 1,000 Articles Taught Me About Working with an LLM
My process for writing technology articles with an LLM, the guardrails that keep them accurate, and why context matters more than any prompt.
I started writing about technology for DZone in 2015. Back then, there was no AI in my process.
Today, my articles are more accurate and informative than what I wrote before, and they need far less editing. Readers, editors, and the people whose stories I tell have been positive about the results.
There's no trick to it. It's a process I've refined over nearly 1,000 articles, and I'll lay it out here. The lesson is simple: the quality of what comes out of an LLM depends on what you put in.
The process, start to finish
Every article I write follows the same steps:
A pitch arrives from a PR or corporate communications person.
I ask Claude to evaluate it. Will my readers care about this? If not, I pass.
I build 10 to 12 interview questions. Claude drafts them from the pitch, the press release, and any documents the company sent in advance.
I do the interview, in person when I can and by video when I can't, and record it with Otter.ai.
I give Claude everything. That means the Otter transcript, my notes, and any reference material: presentations, press releases, my own observations from the room.
I ask for the full package. An 800- to 900-word article, a 150-character meta description, hashtags, social posts for Twitter, BlueSky, and LinkedIn, and a cover image.
I edit. Then I post the article to the publication's CMS.
I share it on Twitter, BlueSky, and LinkedIn using the posts and hashtags from step 6.
The process hasn't changed much. What has changed is how little I have to fix.
Context is the whole game
I've used AI writing tools since November 2021. I started with Jasper, moved to WriteSonic, and now use Claude. Over that time, I went from prompt engineering to context engineering.
Prompt engineering is about how you phrase a request. Context engineering is about what the model knows at the start. I give the model everything I can: the pitch, the transcript, the slides, my notes, and written instructions on audience, voice, and what to avoid. The more it has, the better the output. I've seen that every time.
The interview questions show it best. Because Claude has the pitch and the company's documents before I draft questions, I make few edits, if any. The people I interview have been positive about the results. Some don't believe me when I tell them I'm not a developer and don't have a technology background.
The same thing happens with the articles. As I approach 1,000 articles with this process, the editing has dropped dramatically. Some of that is the tools improving. A lot of it is better instructions and better inputs from me.
Know who you're writing for
One instruction matters more than the rest. I tell the model who the reader is, and the answer is rarely "everyone."
Writing for developers, engineers, and architects is different from writing for IT business professionals. The difference is subtle, but readers feel it. I can write two articles on the same topic for those two audiences, and they won't read alike.
For technical readers, the rules are simple. No marketing speak. Developers hate it. The goal is to make the subject easy to understand and to make their work simpler.
That standard applies to the people I interview, too. If a guest can't explain what their product does in terms I can follow, I say so in the interview. I'm not a developer, so if I can't understand it, my readers won't either. The model can't fix an explanation the source never gave me.
The quote that wasn't there
A few years ago I was writing an article about Groq after a presentation on an IT Press Tour. The draft included a quote that sounded right. It wasn't in anything I had given the model.
I deleted it. Then I asked the model for the source. It admitted it had made the quote up.
That moment shaped how I work. A good draft isn't proof of an accurate one. These are the guardrails I use now:
Record every interview. Otter.ai gives me a transcript, so every quote traces back to what someone said.
Limit the model to my sources. Interviews, whitepapers, press releases, keynotes, and presentations. Nothing else.
Challenge anything suspect. When something doesn't look right, I ask the model where it came from and check the answer against the source.
Run a final quality check. Grammarly keeps the writing consistent before I publish anything.
A fabricated quote under my name would cost me far more than any of these steps.
What stays with me
I can't think of a task I wouldn't share with a model and challenge it on. But I always evaluate the output. It has to make sense, sound like me, and fit my brand.
I also think you need experience doing a job yourself before you hand it to an LLM. I've written hundreds of articles, interviewed thousands of people, and covered several hundred keynotes. I learned this job without these tools. That experience is how I know when a draft is off. It's also why I don't use an LLM to write code. I'm not a coder, and I couldn't judge the result.
There's one line I haven't crossed. I don't let an agent send email on my behalf. That may change as agents improve. Right now I'm testing them on a different part of the job, getting the finished content out to readers.
What this means if you lead content or communications
The editors I write for know I use AI. They keep asking for more articles. The people I interview and the PR and communications teams who bring me their news have been positive about the results. In my experience, nobody thinks much about how an article was made, as long as it's good.
If you work in PR or corporate communications, a few things from my process apply to you:
Send the documents early. Pitches with a press release, a deck, and background material get me better interview questions.
Prepare your spokespeople to explain, not pitch. I write for readers who dislike marketing language. A guest who can describe the work plainly makes a better article.
Expect your own LLM results to track your inputs. A friend of mine has covered technology for more than 25 years. He can't get the quality he wants from his LLM. I don't know whether it's the model or the context he gives it, but I'd bet on the context.
My advice is to give the model everything it needs, work with it over time, and keep a person responsible for the final result. Context is everything.
If you want content built this way, or want help getting more from your own LLM, reach out.
Comments