Representation and memory · Series recap
Beyond tokens: can more efficient AI use less energy and water?
In my four-part series on representation and memory, I explored how AI can represent information, retain history, and carry state forward. That brings me to a question: if an AI system can work directly with numerical representations and retained history, why make it repeatedly turn everything into words?
I believe we can reduce the computation needed for useful work by cutting unnecessary token generation. If that lowers electricity consumption, it also reduces the heat that cooling systems must remove. At facilities that use evaporative cooling, this can mean less water demand.
That is the possibility I want to investigate. I have not demonstrated energy or water savings for Neuralese, memristor-inspired AI, or ECI. My goal is to achieve the same useful outcome with less computation and measure what that changes for energy and water.
15 min read
Practical Takeaway
Keep language where people need it
Use representations to compare, history to preserve evidence, and measured state when it adds value. Generate an explanation when it helps a person act. Judge the design by energy per reliable result and total resource use, rather than token count alone.
Series recap · 01
What I explored in the four articles
In Part 1, I introduced Neuralese as a way to describe machine-native learned representations. An image embedding can encode useful visual information without describing it in sentences. Neuralese is not a standardized protocol, and representations still require computation.
In Part 2, I explored memristor-inspired stateful AI: persistent numerical state that changes with prior activity. Reinforcement, decay, and saturation can be studied in software. Such experiments do not establish the energy efficiency of physical memristor hardware, and extra state can add cost.
In Part 3, I describe Environmental Change Intelligence: repeated observations of the same place, aligned images, learned features, contextual evidence, and longitudinal history. I still need to validate the research platform in the field. Its historical record can be useful without experimental memristor-inspired state.
In Part 4, I separated representation, stored history, and persistent computational state. They can cooperate, but they serve different purposes. That distinction gives us a way to ask which operations an application actually needs.
Read the series in context
I published this recap on September 10, with Part 3 scheduled for September 13, 2026. You can follow the four original articles for the full technical discussion.
Series recap · 02
Getting away from unnecessary token generation
Text tokens are units such as word fragments that a language model reads or generates. In typical autoregressive generation, producing a longer answer requires additional decoding steps. But the model already computes with numerical vectors internally; it does not literally think in English throughout every layer.
I want to explore how much work we can avoid when we stop requiring language between every processing stage. A vision system can compare embeddings, retrieve past observations, and calculate a change score without asking a language model to narrate each step. A person can still receive a concise explanation at the end.
By getting away from tokens, I do not mean that we must abolish every kind of token. Vision models may use image-patch tokens, and latent reasoning systems still perform neural computation. Hiding a long generated explanation from the user also does not remove the work used to produce it. Fewer visible words are an unreliable energy metric.
Series recap · 03
What research supports—and what remains open
Luccioni, Jernite, and Strubell’s FAccT 2024 study found that general-purpose generative models used substantially more inference energy than task-specific models across several evaluated tasks. The results support matching a model to its task; they do not provide a universal savings factor for every current model or deployment.
Coconut research explores feeding hidden representations back into a language model instead of decoding every intermediate step into text. This illustrates a possible route beyond language-only intermediate reasoning. It does not establish that any arbitrary latent architecture will reduce end-to-end energy, nor does it measure water savings for ECI.
My next step would be a controlled comparison: a language-heavy pipeline versus a representation-and-history pipeline on the same observations, hardware, and quality requirements. I would measure the energy savings rather than assume an architecture will deliver them.
Series recap · 04
A persistent memory state for a continuous conversation
The system I have in mind receives a continuous stream, updates a compact memory state, and applies context, preferences, and relevance filters before deciding whether to respond. Instead of reconstructing the whole conversation for every query, it would preserve selected information across time. Language would remain an output interface when a response is useful.
I would start by combining the latest audio features, previous state, user-approved preferences, and an attention gate in each state update. Preferences and contextual priorities are not the same thing as changing learned model weights and bias parameters. Updating those parameters during live conversation is a separate online-learning problem with stability, correction, and forgetting requirements.
I see this as brain-inspired in its continuous perception, selective attention, and retained context. I am not claiming it reproduces the biological brain or establishes human-like understanding. Conventional language models already work with vectors internally: text tokenization and detokenization are usually a small share of inference cost compared with repeated neural computation and memory traffic.
Full-duplex speech research such as Moshi already explores listening and speaking in parallel. It still uses encoded audio tokens and a language-model backbone. Conversational timing, direct audio processing, persistent memory, and token-free computation are distinct design choices; one does not automatically deliver the others.
For an always-listening assistant, I would try a low-power local detector that activates more expensive processing only when needed. Even then, continuous sensing, encoding, state updates, and false activations consume energy. I would compare its energy use over an entire day, including quiet periods, with a push-to-talk baseline. I would also count unwanted interjections alongside helpful ones.
Listening needs a clear boundary
I would want you to choose when listening is enabled, see a clear listening indicator, mute it easily, and inspect, correct, or reset its memory. I would favor short local buffers and explicit retention choices; a persistent state can retain sensitive information even without a saved recording.
# Conceptual stateful interaction, not a trained implementation.
features = encode_audio(stream_chunk)
state = update_state(state, features, approved_preferences)
if response_policy(state).should_speak:
respond(state, retained_evidence)
Series recap · 05
Could a kernel scan language like an image?
My starting point is the way image convolution applies filters across local neighborhoods: early features can respond to edges or textures, while deeper combinations capture more complex patterns. A one-dimensional convolution can similarly slide across a sequence of language representations and produce local feature signals.
Yoon Kim’s 2014 sentence-classification work is an established example of CNNs operating over word vectors. For the assistant I have in mind, I would train a lightweight filter bank to flag local cues related to requests, corrections, preferences, or relevant topics. Those signals would feed the memory update and response policy; they would not independently establish intent.
For example, “I prefer quiet mornings” could provide a candidate preference signal, but “I do not prefer quiet mornings” must not store the same preference. Quoted speech, sarcasm, speaker identity, and references to earlier conversation require context beyond a short window. Wider or stacked filters and persistent state may help; each needs evaluation.
A text CNN usually still starts with words, subwords, characters, or bytes encoded numerically. It can avoid generating long intermediate explanations without being literally token-free. Direct audio features offer another input route, but also require encoding. What I want to measure is how much computation the complete pipeline needs to do useful work.
I would test a small causal convolutional front end plus state against the same system without the filter. Causal means it uses only information already received. I would measure missed requests, false interruptions, latency, and whole-day energy, including the cost of scanning when nobody needs assistance.
# Proposed streaming feature extractor; learned parameters fixed here.
window = recent_input_features() # Past and present only
signals = causal_filter_bank(window)
state = update_state(state, signals, approved_preferences)
action = response_policy(state) # Wait, ask, or respond
Series recap · 06
Which approach is most efficient?
For a narrow, well-defined task, I would start by benchmarking a small task-specific model or simple rule. I do not yet have comparable end-to-end evidence that establishes a winner for the continuous-memory assistant I am proposing. The approaches below also overlap: ECI can use an encoder, persistent state, and eventually specialized hardware.
My research preference is to explore memristor-based computing because bringing memory and computation together could reduce data movement. It is a promising direction, not yet a demonstrated best solution for this assistant. Physical memristor processors already exist in research, including a 2025 mixed-precision memristor/SRAM processor. What I want to build and validate is the complete streaming, adaptive, conversational system I have described here.
Device-level efficiency is not the same as system-level efficiency. I would count conversion circuitry, control logic, memory writes, calibration, and any work still performed on conventional processors. A software simulation of memristor-inspired state does not inherit the energy savings of a physical circuit.
| Approach | Efficiency assessment | Evidence / readiness | Main limitation |
|---|---|---|---|
| Rules or small task-specific model | Best first baseline for a narrow task; not a universal winner | Established; task-specific inference has measured advantages in published comparisons | Limited scope; must meet the same quality requirements |
| Conventional token-generating LLM | Useful for broad language tasks; repeated long generation can be costly | Widely deployable | Cost depends on model, context, reasoning steps, hardware and batching |
| Direct representations / Neuralese-like processing | Can avoid unnecessary language stages | Embeddings are established; Neuralese is a conceptual umbrella | Vectors still cost energy; compatibility and task quality need testing |
| ECI with retained history | Candidate for avoiding repeated interpretation of observations | Research platform; field validation needed | No measured energy or water advantage claimed here |
| Persistent state in software | May reduce rebuilding context; may also add work | Stateful methods exist; proposed assistant needs evaluation | Update, storage, drift and correction costs |
| Physical memristor / analog in-memory hardware | Promising research direction for efficient matrix operations | Research chips exist; full proposed assistant is unproven | Peripheral overhead, precision, endurance and integration |
| Continuous latent reasoning | Potential to reduce intermediate language generation | Research systems such as Coconut | Fewer text tokens do not prove lower total energy |
What would justify naming a winner?
Run the same representative workload at matched quality and latency. Report total joules per accepted result, daily idle/listening energy, and measured water boundaries. I cannot rank their energy efficiency from the architecture names alone.
Series recap · 07
An ECI example: monitor change without narrating every frame
Imagine repeated photographs of a riverbank. A language-heavy design might send every image and a growing history to a large model, request detailed descriptions, then ask another model to compare those descriptions. This repeats encoding, decoding, and context processing even when little has changed.
I would instead check image quality and alignment, compute a representation, and compare it with versioned historical evidence. A small detector or temporal rule identifies observations that deserve review. Routine observations receive structured records; selected findings receive a generated explanation grounded in the saved evidence.
I am proposing this workflow; I have not measured its energy savings in ECI. I would evaluate missed changes, seasonal variation, confidence calibration, and human-review load. Sampling routine results for review helps reveal failures that an escalation threshold would otherwise hide.
# Illustrative workflow; thresholds require field validation.
observation = capture_and_validate()
features = encoder(observation.image)
evidence = history.retrieve(observation.location, encoder.version)
finding = compare_change(features, evidence)
history.store(observation, features, finding)
if needs_review(finding) or selected_for_quality_audit():
queue_human_review(observation, finding)
# Optional language output, grounded in retained evidence:
explain_for_reviewer(finding, evidence)
Series recap · 08
How less computation can mean less cooling water
Servers turn most of the electricity they consume into heat. Reducing their energy use can reduce the heat-removal workload. DOE guidance connects cooling-tower water demand to equipment heat loads and the efficiency of the cooling system. That is the physical connection behind my interest in more efficient AI.
The link depends on the facility. Evaporative cooling consumes water as it rejects heat; other designs use dry cooling or switch between methods. A recirculating liquid loop at the chip does not, by itself, tell us whether the facility consumes water to reject heat outdoors.
There is also an electricity-supply footprint: some power plants use water. DOE warns that a cooling choice that saves water on site can increase electricity use and shift water demand upstream. I would account for both the facility and its electricity supply. Water withdrawn and returned is different from water consumed, such as through evaporation.
A conditional environmental benefit
Fewer tokens do not guarantee fewer kilowatt-hours, and fewer kilowatt-hours do not map to a fixed number of liters. Measure the workload and use the facility’s actual cooling and electricity data.
Series recap · 09
A simple accounting example
Suppose two pipelines process the same 10,000 observations at equivalent detection quality. I am using illustrative values to explain the arithmetic, rather than measurements from my work or a provider: the baseline uses 100 kWh of IT electricity, and the alternative uses 60 kWh.
Assume a constant power usage effectiveness (PUE) of 1.2 and a site water-use intensity of 0.5 liters per IT kWh over the same boundary. The model then estimates 48 kWh less facility electricity and 20 liters less site water use. It excludes electricity-generation water, manufacturing, and construction.
This proportional example is an accounting approximation. Fixed loads, operating conditions, and changing cooling modes can make actual savings different. Do not multiply site liters per IT kWh by facility kWh: those are different denominators.
IT energy saved = 100 - 60 = 40 kWh
Facility energy saved ≈ 40 × 1.2 = 48 kWh
Site water use avoided ≈ 40 × 0.5 = 20 liters
# Illustrative assumptions, not a measured claim.
# Compare equal workloads at equal accepted quality.
Series recap · 10
What this could mean for the environment
If an efficient design displaces electricity generation with emissions, it can reduce the operational climate footprint. The effect depends on where and when electricity is used and which generation responds to demand. A lower bill alone does not quantify avoided emissions.
Where freshwater consumption falls, the benefit can include less pressure on local supplies shared by households, agriculture, and ecosystems. Location and season matter: the same volume has different consequences in a water-stressed basin during drought than in a water-abundant region.
Operational savings also do not erase the footprint of producing chips, manufacturing servers, constructing facilities, or replacing devices. Running inference on a phone can move work out of a data center, but a fair comparison includes device energy and any additional hardware requirements.
This is why I want to pursue the idea: if we can do useful AI work with less total energy and less pressure on local water supplies, we can reduce its environmental impact. I want to keep the measurements that show whether we actually achieve that.
Series recap · 11
Efficiency must reduce totals, too
The IEA’s 2025 Energy and AI report projected data-center electricity consumption of about 945 TWh in 2030 in its base case, more than twice its 2024 estimate. That is a dated scenario for all data centers, not a measurement of token generation or a prediction specific to this series.
An efficient application can still increase its total footprint if use expands faster than efficiency improves. If energy per observation falls by 40% while observation volume doubles, total IT energy rises by 20%: 0.6 × 2 = 1.2. That is why I would track both resource use per accepted result and total monthly consumption.
Turning efficiency gains into environmental gains requires operational choices: avoid redundant runs, set useful capture intervals, limit unnecessary retries, reuse validated results, and retire work that no longer serves a decision.
Series recap · 12
How I would test the idea
Here is the sequence I would use to find out whether the idea works.
- Define a useful result: detection quality, false-negative limits, latency, and the evidence a reviewer needs.
- Compare a language-heavy baseline with a direct representation-and-history pipeline. Add persistent state as a separate experiment so its contribution can be measured.
- Meter the complete workload: encoders, retrieval, storage, state updates, language generation, retries, and idle capacity. Record hardware, model versions, batch sizes, and observation volume.
- Account for training and migration costs. A new model must save enough operational energy over its useful life to repay any additional development footprint.
- Track joules or kWh per accepted result alongside total energy. Ask the operator for site water metrics, cooling method, electricity-source information, and relevant geographic and seasonal boundaries.
- Publish quality and resource results together. Keep the simpler design if added latent reasoning or persistent state does not produce a worthwhile measured improvement.
The direction worth pursuing
My goal across this series is practical: preserve useful representations and history, reduce unnecessary translation into language, and spend computation where it improves a decision. If we can show that this reduces total energy and water demand, we gain more than a faster application. We reduce the resources it takes to run it—and that is the environmental benefit I want to work toward.