# Deconstructing the DuckDB Buffer System.. Part 3

The final components that are yet to be dissected are `blockhandle.cpp` and `bufferhandle.cpp`(here's a [link](https://github.com/duckdb/duckdb/tree/v2.0-cyanoptera/src/storage/buffer) to it). The core aspects of the buffer to be understood by these files are the lifetime and the ownership relations(which essentially overlap) between the objects that constitute the buffer system. I feel like the understanding of these components really wraps up our mental model of how the buffer system works.

### Ownership:

![](https://cdn.hashnode.com/uploads/covers/6a9c0a2185b7af57ae3ad6a6/522d0062-f223-4ea1-b692-1f7baeea3fb4.png align="left")

Here are some premises that helped me understand the ownership aspect of the components better.

*   X owning Y implies that the destruction of the former will cause the destruction of the latter. X having access to Y means X has access to the allowed members of Y.
    
*   `optional_ptr` is a DuckDB-defined pointer. It is a nullable, non-owning ptr.
    
*   The members `memory_p` and `&memory` serve the role of giving ownership and access to a BlockMemory, respectively, to BlockHandle. `memory` is an alias, not a second object.
    

Proceeding with the deconstruction, there were many questions I encountered. After much clarification and rigorous questioning, the following is what my understanding crystallised into:

> Who other than BlockHandle can own a BlockMemory object?

From our current knowledge of the system, only BlockHandle, and possibly other objects that acquire a copy of `memory_p`, can own a BlockMemory object.

> What is a `FileBuffer`? What is the difference between a `FileBuffer` object and a Block?

A block is a fixed-size unit/region that can represent ephemeral/intermediate data or data from persistent storage.

When DuckDB needs, say, Block X, it cannot directly operate on the disk bytes efficiently. It loads that block into RAM. That **chunk of RAM containing the loaded bytes** needs an object to represent it and provide other objects with functions to interact with this chunk. That object is a `FileBuffer`**.**

Summary:

A block is a fixed-size unit/region that can represent an ephemeral/ intermediate data or data from persistent storage. A `FileBuffer` is an object representing the RAM buffer that holds the block's bytes when that block is loaded into memory, along with metadata and operations related to that buffer.

> What is the difference between FileBuffer and BufferHandle? If FileBuffer is already the representative of the buffer in RAM occupied by a block, what is BufferHandle for?

When a reader requests access to a block, that access is granted using an optional\_ptr to the FileBuffer containing the block. This ptr is called BufferHandle. Pinning increases the reader count, while unpinning decreases it.

![](https://cdn.hashnode.com/uploads/covers/6a9c0a2185b7af57ae3ad6a6/e1217efd-5a45-4a92-bccd-c02b97465f75.png align="left")

> When a reader requests access, why is a shared\_ptr to the BlockHandle given along with the optional ptr(BufferHandle) required to access the FileBuffer which holds the block?

Readers, using the part of BufferHandle that contains a shared\_ptr to BlockHandle, collectively own the BlockHandle. The collective lifetime of the BufferHandles thus corresponds with that of BlockHandle and consequently, FileBuffer or the block itself.

* * *

### Lifetime:

The following are some premises that I uncovered in the process of deconstructing these components:

*   `block_id`s are given to blocks based on whether or not they represent regions from persistent storage or intermediate/short-lived data generated during active query execution (e.g., overflow data from a massive sort or join operation). A threshold is defined(`MAXIMUM_BLOCK`). Blocks with block\_ids less than this threshold represent the previous type, while those with IDs greater than the threshold represent the latter.
    
*   Readers are granted access to a block using `BufferHandle`(an optional pointer is a part of the `BufferHandle` object). BufferHandle also has a shared pointer to `BlockHandle`. The collective lifetime of the BufferHandles thus corresponds with that of `BlockHandle` and consequently, `FileBuffer` or the block itself.
    
    <mark class="bg-yellow-200 dark:bg-yellow-500/30">Assumption: BufferHandles are the only strong owners of a </mark> `BlockHandle`<mark class="bg-yellow-200 dark:bg-yellow-500/30">.</mark>
    
*   `CanUnload()` is a function that determines whether a block can or cannot be unloaded.
    
    Its conditions are:
    
    1.  The block must currently be loaded.
        
    2.  The reader count must be zero.
        
    3.  If `block_id` > `MAXIMUM_BLOCK`, the block is only flagged as unloadable if there exists a temporary directory to store this block. Otherwise, it returns false.
        
*   When something needs to read or write to a specific block, a BufferHandle is provided to this fn or user. This loads/pins it, which increments the reader count. Unpinning decrements it and further triggers a check of the reader count. If it becomes zero, this block is pushed into the eviction queue. When memory pressure occurs, `CanUnload()` is run on this block. If this returns true, it may then be unloaded.
    

A few more questions that I encountered in the process of this deconstruction:

> Is getting access to a FileBuffer the same as getting access to the block which resides in that buffer?

`GetFileBuffer()` **itself does not magically give you a high-level "Block object."** It gives you the underlying byte storage. The code that knows the block's structure interprets those bytes as the block's actual contents.

> Is a decrement in the reader count the sole consequence and purpose of unpinning? Same question for Pinning(is the increment the sole purpose?).

Pinning loads the block into RAM(if not already loaded), increments the reader count, and returns a BufferHandle to the requester. Unpinning reverses all of this.