00-project-skeleton
I am very much on the learning curve for how to apply AI for development in the form of Large Language Models (LLMs).
An LLM has in essence read and distilled the entire web, and most(?) printed books. On one hand, this is an astounding breadth of knowledge. But not deep knowledge. If you ask the right question, that question will match fragments. The LLM will then (lightly) lace together those fragments, and the result can save you days or weeks of research. This is astounding, but slightly unreliable.
LLMs require prompts, which turned out to be familiar. For decades, I have written whitepapers, architecture documents, and endless conversations and emails - all with the intent of getting coworkers "on the same page". Turns out this activity - aimed at other humans - is very much like writing prompts for an LLM.
While LLMs have made extraordinary progress at understanding unstructured natural-language text, there is nothing like standardized benchmarks. This means that each LLM might interpret a set of prompts in not-quite the same manner. Makes writing prompts ... just a bit tricky.
At this point I have gone through several iterations, writing prompts, evaluating the results, and coming up with better. Prompts can define the conventions and workflow for a project. So you are describing the frame for working on a project, as well as the detail specific to the project.
On the one hand, an LLM has access to the sum total of human knowledge (or nearly). Given the right questions, an LLM can retrieve - and lightly lace-together - from near everything known to human mind. On the other hand, an LLM does not have deep knowledge, and so can sometimes lace-together stories that are nonsense.
What that means in present is that for a human mind an LLM can save enormous time, as pulled-together stories can save days or weeks of research. At the same time, a strong human mind is needed to recognize and reject nonsense stories.
My analogy is that an LLM is very much like an amplifier in electronics.
- An amplifier has:
- an input signal
- output gain, noise, and distortion
- The input signal to an LLM is prompts written from the human mind.
- A strong, well-ordered mind can write prompts that provide a strong, clean signal.
- A lesser mind offers a weak, noisy signal.
Over the last several years, the gain offered by an LLM has improved, while the noise has dropped. But the distortion has not gone away.
The interesting aspect here is that you can use the LLM to help define the workflow for working with an LLM. Ask an LLM to suggest a workflow, and the LLM will pull together a workflow from the sum of current human knowledge. Ask an LLM for conventions, and the LLM will pull from the sum of knowledge. Makes a good start.
Working with LLMs has made me aware of the set of rules in my head compiled from five decades of experience. Those rules have evolved with time and experience. Have tried to capture those internalized rules.
Compiled an example project skeleton from the sum of human knowledge, filtered through internalized rules.
It is at least a start. :)