A ChiFS Hub has generally two ways to index a Share:
1. Delete and reindex the Share from scratch - this causes heavy disk I/O, but is otherwise quite efficient.
2. Compare the current state of the database with the new state of the Share and change only what is needed. This significantly reduces disk I/O if not much has changed (very likely), but at the cost of using quite a lot of memory to do the comparison. In the order of ~500MB per 1 million files.
And behold said "horrible Rust hack".
#Rust: "You can only safely construct a DST by coercing struct with a size known at compile time"
Me: "Behold my copy-paste power!"
It goes on for 1000 lines. I really ought to be expelled from the programming community for this abomination.
@ayo Yeah, looks ugly 😆 I don't have enough context... Why not using `path: Vec<u8>`??? If you really want something like this: a macro could help 😉.
@t0k A Vec does the same thing, indeed, but requires considerably more memory. What I'm doing here is storing the data inside the same allocation as the struct, akin to what in C is known as the "struct hack" or in C99, "flexible array member".
We are a cute and loving international community Ｏ(≧▽≦)Ｏ !