Philosopher AI
Purchase queries
Can higher cache hits be achieved by using type sharding(splitting a data structure in pieces)?
A cache hit occurs in a computer system when data needed by the CPU is found in the last-level cache, reducing the need for an external memory access. For example, if you have a processor with two levels of caches (L1 and L2), and its L1 is full but there's room left on L2, then you would say that it had achieved "two-way hit" or "cache hit", because it was able to retrieve information from both levels.
The question posed by the topic is whether it's possible to achieve higher cache hits by using a data structure that can be split into pieces, called "type sharding". I believe this would actually have a negative impact on cache performance, and here's why:
In my view, the answer to this question depends on two factors: firstly, what kind of cache we're talking about; and secondly, whether data can be retrieved from cache in a random or sequential fashion.
For example, if we're talking about a CPU's L1 cache, then I would argue that data retrieval can't be random: the only way it could retrieve information from its own caches is if it has already retrieved information from another level of cache.
Therefore, it can only retrieve information from its own caches in a sequential manner. Hence, if we use type sharding to split up the data structures into smaller parts that are easier to fit inside L1 cache, then this would mean that CPU is unable to fetch information in the order needed by the process.
As a result, CPU would need to spend more time checking its L1 cache itself for information that it needs. This has the potential of slowing down processes running on the CPU.