What software engineers bring to AI-assisted software development

Getting professional-quality results while using an LLM for programming is possible, but it takes practice and most of the same skills it takes to write professional-quality code. This post documents what I feel are the most important skills & qualities a software engineer brings to the table while using an LLM for programming.

Domain knowledge

LLMs don’t understand your business. They don’t understand the problem you’re solving. It’s your job to understand the domain and use that understanding to drive the model.

Software engineers must have deep understanding of the domain and of the specific problem they’re solving. Often, they already have the shape of a solution in mind. It’s the SWE’s job to communicate that knowledge to the LLM to allow it to create the solution.

System knowledge

LLMs don’t know about the existing system(s) that any particular codebase interacts with. A software engineer aiming to produce professional results must understand what systems exist now, how they interact, and where this change fits in. They need to clearly communicate this understanding to the LLM to ensure the change fits neatly into existing systems & solutions.

Understanding & clarifying requirements

This one isn’t LLM-specific (really, very few of these items are).

One of the hardest part of software engineering is translating often-ambiguous English requirements into cold, pure logic. LLMs don’t absolve you of this responsibility! It’s an SWE’s job to thoroughly understand the requirements for the problem you’re trying to solve and clarify as needed. Then, and only then, can you guide the LLM in the correct direction.

Product sense

Software engineers — and humans more generally — bring a sense of how their product works and where and how a new feature fits into it. LLMs don’t have that sense, and asking an LLM to implement some feature somewhere is unlikely to result in an optimal experience. Providing an LLM with clear direction on where and how to implement a change is critical for attaining good results.

Correct and valuable context

This item is intentionally broad — it encompasses all the previous points, and more.

A key insight is that the quality of results from an LLM depends on the context you’ve provided it with. This includes writing prompts with specific requirements, a clear direction on where and how to implement them, and well-specified end goals.

It also includes the tools available to the LLM. These might be research tools, an ability to examine issues and code directly from GitHub, or a specialized feature development skill. See my post on giving your LLMs tools for more.

For more on “context engineering”, see this post from Simon Willison and this one from the Langchain blog.

Taste

A professional SWE will have developed a sense of what makes good code, in terms of code style, readability, naming, and architecture (to name a few). They should bring that taste into the LLM-assisted development process, and ask the LLM to iterate when it produces code that isn’t up to par.

A simple example might be “inject this dependency when this struct is created; don’t pull it in from over there.”

Architecture guidance

I’m calling this out as distinct from “taste” even though it’s implied there. LLMs are not architecture experts; they might accidentally follow a good pattern, particularly when working in an established, well-architected codebase. But more often than not, they’ll benefit from concrete guidance.

After getting an initial result from an LLM, it’s important to review its architecture and refactor it until you’re proud of the code quality. Whether you refactor using the LLM or by hand (ideally using the tools built into any powerful IDE) is up to you.

Validation and testing

This one goes hand-in-hand with “understanding & clarifying requirements”. Your job is to deliver code you’ve proven to work. A professional SWE will carefully review results from an LLM and validate them — ideally by adding to the test suite — to be sure they fulfill well-understood requirements.

Pushback

Finally, just as when working with another engineer, a professional software engineer will sometimes push back on an LLM. LLMs will make assumptions and assertions; it’s your job to check them and question them when something seems off.

Conclusions

LLMs don’t mean the end of software engineering as a profession, just like myriad other technologies didn’t either. The skills & qualities outlined here — among others, I’m sure — are unlikely to be satisfactorily replaced by an LLM, certainly in the near future. Based on my own experience using LLMs for both hobby and professional software development, I strongly believe that they’re essential to producing professional-quality code.

Obie Fernandez has also written on this subject; I recommend also reading his post, “What happens when the coding becomes the least interesting part of the work”.