What should a knowledge worker be doing to prepare?
Some concrete and pragmatic recommendations
A fast-follow to a question I raised at the end of the Deep Research post.
Use the tools (better)
Increasing fluency with AI tools is on obvious place to start. I think there is a tremendous difference in skill and comfort with AI tools between people who have 10+ hours of experience with AI tools and those that don’t. There may be a similar almost step-function change at 100+ hours. I had previously predicted that I thought that prompting as a skill was going to become less important over time. While in the long run perhaps this will still be true, currently prompting continues to go up in value. In addition to time spent engaging with the tools, differences in approaches to prompting and prompting skill I believe contribute to some of the wide differences in opinion as to how useful these tools are.
Other than experimenting and using the tools, the best advice I can think of when it comes to prompting is to keep providing more context than you think is reasonable. If your default prompting is a few words, try a few sentences, if it’s a few sentences make that a few paragraphs, if it’s a few paragraphs already, then think about what background documents might be helpful or begin refining your “custom instructions” so that all of your results might be more useful to you. Put another way, if you’ve never run into a prompt where you wished you had access to a larger context window, you’re very likely not getting the most of these systems. The primary time I use Gemini models is when I want an absurd context window, but it’s great that it’s there when you want that feature.
Part of the difficulty of building skill here is that the breadth of the interface is unbelievably large (most of human language). So it takes time to learn how to even think of the things you might want to ask for. Here’s a short list of some ideas:
Use simple sentence structures.
Avoid specific words or themes.
If you'd like the model to emulate your writing style, provide writing samples.
Request CSV output explicitly, tell the model that you don’t need additional commentary.
Ask the model to replace typical apologies with light-hearted jokes. (This technique worked exceptionally well with Claude Sonnet 3.5, which tended to be overly apologetic.)
Provide detailed examples, negative examples, or multiple examples to clearly illustrate your instructions.
You can also explore novelty prompts, such as requesting responses where every word starts with an "s," or responses that rhyme every other line. These prompts help demonstrate the model’s flexibility and compliance with arbitrary instructions rather than producing practical outputs.
Do some emotional work
A much harder recommendation is to check your ego1. I believe a lot of the resistance to using AI tools is from feeling threatened. This is a totally normal response, I get it, but it’s also not helpful. It’s a deeply human solution to ego protection to de-value and denigrate that which threatens us. The more openness that knowledge workers have to the possibility that the models provide value, the more they will be able to see and capture that value.
There’s more to explore here. I think how we relate to our labor, ourselves and each other are critical areas to understand. I hope to think more about this side of things in the future when I have more concrete thoughts around policy.
Shift your sense of what’s valuable
Another thing that knowledge workers can do to prepare is to think critically about what will be of higher value and lower value in the near and intermediate future and prioritize efforts accordingly. This will probably vary by industry and also be potentially volatile (AI tool progress will likely be non-linear and unpredictable, so the occasional unexpected advanced capability arriving years earlier despite some much easier thing being elusive should be the expected norm), but here are some high level thoughts.
Surface level knowledge and very deep knowledge will both go up in value, while intermediate knowledge will go substantially down in value. This will be incredibly disruptive because most of the value that most knowledge workers provide most of the time in the current paradigm is at this intermediate level. There’s a lot to unpack here, so let’s dig into the terms a bit better so that the concepts are more clear and then get into why I think this. I’ll also end up using some examples from software engineering because that’s what I’m familiar with, but I think that these principles will generalize reasonably.
Surface level knowledge is 0-15th percentile understanding. It’s just enough to know that a thing exists, how it might useful and to ask a handful of follow up questions to learn more.
Intermediate level knowledge is 15th-85th percentile understanding. My contention is that the majority of knowledge work takes place in this bucket today. It’s understanding well enough to do the implementation and produce something. It falls well short of novel research or advancing the field, but is the consistent application of an established body of knowledge to new problems.
Deep level knowledge is the remaining 85th-99th+ percentile understanding. This contains the difficult tasks where there is no paved road. Perhaps the complexity of the situation is just enough to make it novel, or maybe no one has needed to literally ever do this thing before. A lot of people imagine that a lot of work happens in this bucket, but I’m pretty skeptical. I don’t work on something like this even once a year in my career and I think I’d have to make a lot of sacrifices and compromises that I don’t want to if it was important to me to do work in this category.
So, why do both ends of the distribution become more valuable, while the bulk in the middle becomes less valuable?
Surface level knowledge becomes more useful because AI tools can much more quickly translate surface level knowledge into intermediate level knowledge. So knowing the space of what’s possible, what questions that need to be asked, how to connect disparate ideas and domains, all goes way up in value. Breadth of knowledge has always been useful, but it has been pretty limited because you either needed to learn the intermediate knowledge yourself to make it pragmatically useful or you needed to recruit the help of someone else more experienced. But now it’s possible to extremely quickly close the gap from 7th percentile to 70th percentile (at least in terms of the thing that is actually produced, it’s a deep illusion that YOUR understanding has increased this much, but with AI tools you can solve problems as if your understanding was at the 70th percentile, how much this distinction matters depends substantially on context2 — we’ll come back to this).
Let’s take a software engineering example: web browsing safety checks for malicious URLs. Here’s a problem where we have a large dataset (millions of known bad URLs) and we want to extremely quickly understand if a URL is in this dataset. We’re in a resource constrained environment and our initial efforts to do this naively are just too slow. We have just enough surface knowledge to know about probabilistic data structures, so we ask an AI model in there is a good solution to our problem. Yes, it turns out a Bloom filter would work perfectly here3, its a space-efficient probabilistic data structure that test whether an element is a member of a set. False positives (we think a URL is malicious, but it’s not) are possible, but false negatives (we think the URL is safe, but it’s actually known to be malicious) are not.
So we had just enough knowledge to ask the question of whether probabilistic data structures could help and we can get a concrete answer and confirmation that it would work well for our problem. And in a typical software engineering ecosystem, all you really need to do then is to track down a library that someone else has already written that does this. But let’s say you’re working on a problem that has analogous characteristics but in a less well developed ecosystem like Rust. Maybe you need to write your own Bloom filter. AI can help with that too — see the appendix for Claude Sonnet 3.7’s first attempt at this.
Implementing a Bloom filter would be at the very high end of complexity for most software engineering tasks that someone actually encounters day to day. The expectation is that a good engineer could do it, but practically it’s just not necessary. And by “do it”, I don’t mean just sit down and from memory, leetcode style, crank one out. I mean, do some research, look for a reference implementation, write some tests, get some feedback from a peer or mentor and several days or a week+ later come back with something.
But now we can go from “hazy notion about there could be something that would help here” to “here’s a concrete implementation of the idea that you can start benchmarking to see if it fulfills latency requirements” almost immediately.
Surface knowledge is so valuable because it increases that chance of asking the right question, seeing a particular connection and just broadly understanding the “what”. At the risk of a strained analogy, I see much of intermediate knowledge as the “how” and deep knowledge as perhaps the “why”. The reason so much much knowledge work is at the intermediate level is because knowing “how” is exactly the path to the repeated execution and application of known principles to slightly different data or situations. When the “how” becomes trivial, knowing “what” goes up in value, but so also does the much deeper “why”.
Why does deep knowledge become more valuable also? Because, at least currently and probably for at least some foreseeable future, it exists beyond what the AI models can accomplish. If we’re pushing the frontier and developing novel probabilistic data structures for new use cases, its entirely possible that AI tools help with this process, but knowledge workers can still provide differentiated value here. Also deep knowledge is critical for the evaluation and understanding of the outputs of those that have surface level knowledge and are using AI to get to intermediate level knowledge output without the corresponding actual understanding. There is a tremendous risk surface area here. The upside is that surface knowledge can translate extremely quickly into intermediate level outputs, but the downside is that the surface level user is not well positioned to understand the outputs and how they relate to other projects and domains. So deep expert level checks to ensure the correctness and contextual appropriateness goes way up in value. This will be the new bottleneck in the short / medium term.
Contextualizing
I sort of hope that I’m wrong about my model of surface, intermediate and deep knowledge. For two reasons.
One is selfish: my comparative advantage is getting to high intermediate knowledge in something very quickly. I excel at going from 0 to ~75th percentile faster than most people. However, it takes about the same amount of effort and time for me to go from 0 to 75th as it does for me to go from 91th to 92nd percentile. Incremental progress at the top end of the distribution is slow, tedious and a bit painful (for me). So, if I’m right about all of this, my personal human capital takes a pretty big hit here. I can perhaps mitigate some of this by following my own advice — I think it will be much easier for me to try to increase breadth rather than depth. And reflecting back on this, I think I’ve already begun doing this. In the past several years, I have many more new interests that I only understand pretty superficially.
The other is concern for the level of disruption that this will cause. A core part of my hypothesis is that the vast majority of knowledge work occurs in the area that is going down in value the most. So if most of what most people have been spending their time getting good at is quickly less and less valuable, I worry about the labor implications. Re-skilling to more surface level breadth seems plausible, but will have have diminishing returns as more people do it. Re-skilling to expert level depth is extremely challenging and unrealistic to be a broadly successful strategy. More to say here when I get around to writing about policy.
But, of course, I also hope that I’m right, because I do really like being right about things. And both emotional and intellectual flexibility will also go substantially up in value.
So how should knowledge workers prepare:
Use AI Tools (better), don’t be emotionally defensive and focus on the extremes of knowledge depth.
Appendix
Links
Manus just dropped, state of the art AI agent or hype? Highly speculative, but I’d guess the former. Much better than Deep Research? We are unprepared for accelerating rate of change.
A Bloom filter in rust
created by Sonnet 3.7 with extending thinking in 1 minute and 18 seconds
use std::hash::{Hash, Hasher};
use std::collections::hash_map::DefaultHasher;
use std::marker::PhantomData;
use std::fmt;
/// A Bloom filter is a space-efficient probabilistic data structure that tests
/// whether an element is a member of a set. False positives are possible, but
/// false negatives are not.
#[derive(Clone)]
pub struct BloomFilter<T: Hash> {
// The bit array represented as a vector of bytes
data: Vec<u8>,
// Number of bits in the filter
size: usize,
// Number of hash functions
num_hashes: usize,
// Phantom data to indicate the type of elements stored
_marker: PhantomData<T>,
}
impl<T: Hash> BloomFilter<T> {
/// Create a new Bloom filter with the given size (in bits) and number of hash functions
pub fn new(size: usize, num_hashes: usize) -> Self {
// Calculate number of bytes needed
let num_bytes = (size + 7) / 8;
BloomFilter {
data: vec![0; num_bytes],
size,
num_hashes,
_marker: PhantomData,
}
}
/// Create a new Bloom filter with the optimal size and number of hash functions
/// for the expected number of elements and desired false positive rate
pub fn with_rate(expected_elements: usize, false_positive_rate: f64) -> Self {
let size = Self::optimal_size(expected_elements, false_positive_rate);
let num_hashes = Self::optimal_num_hashes(size, expected_elements);
Self::new(size, num_hashes)
}
/// Calculate the optimal size for the bit array
fn optimal_size(expected_elements: usize, false_positive_rate: f64) -> usize {
let size = -((expected_elements as f64) * false_positive_rate.ln()) / (2.0_f64.ln().powi(2));
size.ceil() as usize
}
/// Calculate the optimal number of hash functions
fn optimal_num_hashes(size: usize, expected_elements: usize) -> usize {
let hashes = (size as f64 / expected_elements as f64) * 2.0_f64.ln();
hashes.ceil() as usize
}
/// Hash the element with the given seed
fn hash(&self, element: &T, seed: usize) -> usize {
let mut hasher = DefaultHasher::new();
seed.hash(&mut hasher);
element.hash(&mut hasher);
(hasher.finish() as usize) % self.size
}
/// Set the bit at the given index to 1
fn set_bit(&mut self, index: usize) {
let byte_index = index / 8;
let bit_index = index % 8;
self.data[byte_index] |= 1 << bit_index;
}
/// Check if the bit at the given index is set to 1
fn get_bit(&self, index: usize) -> bool {
let byte_index = index / 8;
let bit_index = index % 8;
(self.data[byte_index] & (1 << bit_index)) != 0
}
/// Add an element to the Bloom filter
pub fn insert(&mut self, element: &T) {
for i in 0..self.num_hashes {
let index = self.hash(element, i);
self.set_bit(index);
}
}
/// Check if an element might be in the Bloom filter
pub fn contains(&self, element: &T) -> bool {
for i in 0..self.num_hashes {
let index = self.hash(element, i);
if !self.get_bit(index) {
return false;
}
}
true
}
/// Get the current number of bits set to 1
pub fn count_ones(&self) -> usize {
self.data.iter().map(|&byte| byte.count_ones() as usize).sum()
}
/// Estimate the number of elements in the Bloom filter
pub fn estimate_elements(&self) -> f64 {
let m = self.size as f64;
let k = self.num_hashes as f64;
let x = self.count_ones() as f64;
-m / k * (1.0 - x / m).ln()
}
/// Clear the Bloom filter
pub fn clear(&mut self) {
for byte in &mut self.data {
*byte = 0;
}
}
/// Get the size of the Bloom filter in bits
pub fn size(&self) -> usize {
self.size
}
/// Get the number of hash functions used
pub fn num_hashes(&self) -> usize {
self.num_hashes
}
/// Calculate the current estimated false positive rate
pub fn false_positive_rate(&self) -> f64 {
let n = self.estimate_elements();
let m = self.size as f64;
let k = self.num_hashes as f64;
(1.0 - (1.0 - 1.0/m).powf(k * n)).powf(k)
}
/// Union this Bloom filter with another (combines both sets)
/// Both filters must have the same size and number of hash functions
pub fn union(&mut self, other: &BloomFilter<T>) {
assert_eq!(self.size, other.size, "Bloom filters must have the same size");
assert_eq!(self.num_hashes, other.num_hashes, "Bloom filters must have the same number of hash functions");
for (i, byte) in other.data.iter().enumerate() {
self.data[i] |= byte;
}
}
/// Intersect this Bloom filter with another (keeps only elements in both sets)
/// Both filters must have the same size and number of hash functions
pub fn intersect(&mut self, other: &BloomFilter<T>) {
assert_eq!(self.size, other.size, "Bloom filters must have the same size");
assert_eq!(self.num_hashes, other.num_hashes, "Bloom filters must have the same number of hash functions");
for (i, byte) in other.data.iter().enumerate() {
self.data[i] &= byte;
}
}
/// Check if this Bloom filter contains all elements of another
pub fn contains_filter(&self, other: &BloomFilter<T>) -> bool {
assert_eq!(self.size, other.size, "Bloom filters must have the same size");
assert_eq!(self.num_hashes, other.num_hashes, "Bloom filters must have the same number of hash functions");
for (i, byte) in other.data.iter().enumerate() {
if (self.data[i] & byte) != *byte {
return false;
}
}
true
}
}
impl<T: Hash> fmt::Debug for BloomFilter<T> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("BloomFilter")
.field("size", &self.size)
.field("num_hashes", &self.num_hashes)
.field("count_ones", &self.count_ones())
.field("estimated_elements", &self.estimate_elements())
.field("false_positive_rate", &self.false_positive_rate())
.finish()
}
}
// Example usage
fn main() {
// Create a new Bloom filter with 10,000 bits and 4 hash functions
let mut filter = BloomFilter::<String>::new(10_000, 4);
// Insert some elements
filter.insert(&"apple".to_string());
filter.insert(&"banana".to_string());
filter.insert(&"cherry".to_string());
// Check if elements are in the filter
println!("Contains apple: {}", filter.contains(&"apple".to_string())); // true
println!("Contains banana: {}", filter.contains(&"banana".to_string())); // true
println!("Contains cherry: {}", filter.contains(&"cherry".to_string())); // true
println!("Contains date: {}", filter.contains(&"date".to_string())); // false
// Get statistics
println!("Estimated elements: {:.2}", filter.estimate_elements());
println!("False positive rate: {:.6}", filter.false_positive_rate());
// Create a filter with a desired false positive rate
let mut better_filter = BloomFilter::<String>::with_rate(1000, 0.01);
println!("Size: {}", better_filter.size());
println!("Number of hash functions: {}", better_filter.num_hashes());
// Demonstrate union operation
let mut filter1 = BloomFilter::<String>::new(1000, 4);
filter1.insert(&"hello".to_string());
let mut filter2 = BloomFilter::<String>::new(1000, 4);
filter2.insert(&"world".to_string());
filter1.union(&filter2);
println!("Contains hello: {}", filter1.contains(&"hello".to_string())); // true
println!("Contains world: {}", filter1.contains(&"world".to_string())); // true
}The irony of this advice coming from someone with a very high opinion of themselves is not lost on me…
This is of critical importance when thinking about the use of AI in education — punting deeper investigation here to a future policy post
The win here is that you can store the URLs locally on a device because of the efficient space compression, which, even after the additional compute of the hashing functions the savings of needing to avoid an external network call will be worth it