Re: [tlug] how to tune reiser4 for millions of files?

[Kalin KOZHUHAROV <me.kalin@example.com>]

Just as last-resort-one-hack-off-the-top-of-my-head...

You can "make your own" filesystem by appending all the data, possibly
with some delimiter. Then use offsets as index to your former files,
but you can drop all the metadata (like filename) inside your file.


What I am suggesting is to concatenate all files in order that you
might need them.

[file1_data]
===nothing-like-this-delimiter===
[file2_data]
===nothing-like-this-delimiter===
...

and corresponding index:
file1_start: 0
file2_start: 34562
....


Of course, nobody mentioned importing that into a normal database :-)
MySQL will be happy to glob your data into a BLOB type, or you can
save your self a good deal of time, if you parse the data from each
file and input it in the database (iff the data is structured of
course), then do:

$ for file in $(find -type f -print ) do; ./my_parser ${file} >>data.csv; done

Kalin.