In programming, reading and writing are how your code talks to the outside world.
You’re dealing with a file, a network connection, or even just printing something to the screen, you’re basically reading bytes from somewhere and writing bytes somewhere else.
It could be a file, a network stream, memory, or even a database, doesn’t matter. Under the hood, it’s always the same idea: you read when you want to get data, and you write when you want to send or store it.
In this post, I’ll walk through:
-
How Read and Write work at a low level
-
A custom MemoryBuffer that implements both traits
-
A step-by-step explanation of file → memory → stdout I/O flow
-
What happens when you use tiny buffers and why that matters
Traits in Rust
In Rust, the Read trait provides:
fn read(&mut self, buf: &mut [u8]) -> Result<usize>
This fills buf with data and returns how many bytes were read.
The Write trait provides:
fn write(&mut self, buf: &[u8]) -> Result<usize>
fn flush(&mut self) -> Result<()>
This writes buf into a sink (file, stdout, memory, etc)
Implementing a Custom MemoryBuffer
Imagine we want to build a custom buffer, basically an in-memory array that supports reading and writing like a file It has a small internal array of 4 bytes and two pointers: one for reading, one for writing.
struct MemoryBuffer {
data: [u8; 4],
write_pos: usize,
read_pos: usize,
}
data: a fixed-size byte array (only 4 bytes!)
write_pos: index where the next byte will be written
read_pos: index where the next byte will be read
We're mimicking how a real I/O buffer works in limited memory. This small size helps us learn how real-world systems handle streaming data chunk by chunk
Implementing Helper Methods
We’ll add some helper methods to check how much space is left to write, how much data is available to read, and to reset the buffer.
impl MemoryBuffer {
fn new() -> Self {
Self {
data: [0; 4],
write_pos: 0,
read_pos: 0,
}
}
fn available_data(&self) -> usize {
self.write_pos - self.read_pos
}
fn available_space(&self) -> usize {
self.data.len() - self.write_pos
}
fn reset(&mut self) {
self.write_pos = 0;
self.read_pos = 0;
}
}
available_data: how many bytes we can read
available_space: how many bytes we can still write
reset: clears both read and write positions (but not the data we just overwrite it)
Implementing Read for the Buffer
impl Read for MemoryBuffer {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
let available = self.available_data();
if available == 0 {
return Ok(0); // EOF or no data to read
}
let to_read = buf.len().min(available);
buf[..to_read]
.copy_from_slice(&self.data[self.read_pos..self.read_pos + to_read]);
self.read_pos += to_read;
Ok(to_read)
}
}
We basically copy bytes from our internal buffer into the caller’s buf. We update read_pos to track how many bytes we’ve already read
Implementing Write for the Buffer
impl Write for MemoryBuffer {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
let space = self.available_space();
let to_write = buf.len().min(space);
if to_write == 0 {
return Ok(0); // buffer full, no space to write
}
self.data[self.write_pos..self.write_pos + to_write]
.copy_from_slice(&buf[..to_write]);
self.write_pos += to_write;
Ok(to_write)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
We take in some bytes and store them in our internal array and we only write as much as we have space for.
Flush doesn't do anything in our case, it's mostly used for buffered output like files or sockets
Let's build the I/O flow
Remember, the flow we want to achieve is: Read data from a file, write it into a memory buffer, read it back out of that buffer, and write it to stdout (our terminal)
fn main() {
let mut file = std::fs::File::open("input.txt")?;
let mut file_buf = [0; 4];
let mut mem = MemoryBuffer::new();
let mut out_buf = [0; 8];
let mut stdout = io::stdout().lock();
loop {
let n = file.read(&mut file_buf)?;
if n == 0 {
break;
}
let mut written = 0;
while written < n {
mem.reset(); // clear buffer to add new data
let wn = mem.write(&file_buf[written..n])?;
written += wn;
loop {
let rn = mem.read(&mut out_buf)?;
if rn == 0 {
break;
}
stdout.write_all(&out_buf[..rn])?;
}
}
}
stdout.write_all(b"\n")?;
}
How It Works
Let’s say input.txt contains:
HELLO_WORLD
-
We read 4 bytes at a time: ['H', 'E', 'L', 'L'], then ['O', '_', 'W', 'O'], then ['R', 'L', 'D']
-
We write those bytes into our MemoryBuffer
-
We read them back from the buffer and write to stdout
-
This repeats until the file ends
Even with a tiny 4-byte memory buffer, we still process the whole file, just in multiple small reads and writes. That’s the core of streaming!
Conclusion
Everything in I/O comes down to reading bytes from somewhere and writing them somewhere else. Whether you're working with:
-
files
-
sockets
-
pipes
-
databases
-
or even in-memory structs
... it's all about how you move data :)