Existing large language models (LLMs) indeed have many practical values. Nicholas Carlini, a research scientist at Google DeepMind, shared in detail over 50 instances of using LLMs in his work, covering various aspects such as programming, writing, and learning new technologies.
Nicholas believes that LLMs are not overhyped because they can indeed handle increasingly difficult tasks. Over the past year, he has spent at least a few hours each week interacting with various LLMs, which have helped him increase his coding speed by at least 50% in research projects and side projects.
Nicholas listed some specific examples of using LLMs:
- Building entire web applications using technologies he's never used before
- Learning to use new frameworks and tools
- Automatically converting programs to C or Rust for improved performance
- Simplifying and reducing large codebases
- Writing initial experimental code for research papers
- Automating monotonous tasks and one-time scripts
- Replacing web searches for setting up and configuring new software
- Helping debug error messages
Nicholas categorizes these applications into two types: helping with learning and automating boring tasks. While these applications may not seem fancy, they all stem from actual work needs and demonstrate the value of LLMs in automating the tedious parts of work.
As a security researcher, Nicholas's work over the past decade has been to demonstrate how AI models might fail in unknown environments. He fully understands the limitations of these systems. However, he still believes that LLMs have brought the biggest improvement to his work efficiency since the birth of the internet.
Nicholas detailed how to use LLMs to build complete applications and learn new technologies. For example, he used GPT-4 to write a mini-game called "GPT-4 Capability Prediction Challenge," with the initial version of the entire application almost entirely completed by GPT-4.
In terms of learning new technologies, Nicholas illustrated how to use LLMs as tutors to learn new tools like Docker. Compared to traditional learning methods, having LLMs directly teach the required knowledge is much more efficient.
Nicholas wrote this article to prove that LLMs have already provided him with a lot of value and to provide some examples for those who don't know how to use LLMs. He acknowledges that LLMs currently can't solve the most difficult and interesting parts of a programmer's work, but they are already good at handling simple tasks, greatly improving work efficiency.
Five years ago, LLMs could at most write seemingly fluent but practically useless text. Now, they can increase Nicholas's programming efficiency by an average of 50%. This progress is impressive and suggests that LLMs may bring even greater changes in the future.