OT: multi-platform *nix blocksize detection

[Fairlight]

On Wed, Oct 20, 2004 at 02:27:42PM -0400, Bill Vermillion may or may not have
proven themselves an utter git by pronouncing:
> 
> > Something is goofy on linux systems.
> 
> If thought that since the first Linux system I saw :-)

You know....*FWAP!*  :)

> I'm looking at a SuSE system now.  On it the /etc/profile
> is 6845 bytes long.    ls -ls gives me 6845 bytes but ls -lk
> gives me 7 - it has allocated 7K.

You're surely not using ext2.  Check your fs type.  You -shouldn't- be.
SuSE likes to default to ReiserFS, and if you don't use that, you'd
probably want to use ext3.  Both of those are journalling, ext2 is not.

> Perhaps during install someone change the default block size
> in mkfs.ext2  or mkfs.ext3 [or whatever] when the file system
> was created.  You can not depend on all systems being installed
> with the default block sizes.

True.  Which is -why- I wanted a surefire way to find the default block
size--preferably without relying on 'dd' and 'ls' to do so.  It's tempting
to look inside the source to 'ls' and find out exactly how it comes up with
its number of blocks.  It seems to be reliable on any system.

> The SuSE 9.1 I looked like is not jibing with the SuSE you
> are indicating.  And shouldn't the file creation be independant of
> the OS, and in particular the system utilities such as ls, df, et
> al shouldn't be kernle dependant but upon the filesystem.

9.1 runs the 2.6 kernel, 9.0 runs 2.4.  This is why I won't use 9.0.  :)

As for your second question...yes, the operation should be
filesystem-dependant, although you have to keep in mind that the VFS layer
is in the kernel, and that's probably where linux's bits/stat.h comment
about 512-byte blocks for st_blocks comes into play.  

What's really weird is that I create a 1024-byte file on Solaris and I get
an 8K reading for blocks.  Funny thing is, if I just echo a few words into
a file (18 bytes), I get 0K allocated.  I wonder where Solaris thinks it's
storing the data, and what the threshhold for using extra blocks is.  What,
is there room in the inode itself for short, tiny things?

> > So it's not even a matter of scaling down to 512-byte blocks,
> > it's a matter of always dividing by 2 to get the right number.
> 
> On that system.  What is there is another block size specified.
> Linux permits 1024, 2048, and 4096.  Systems such as FreeBSD
> default to 16384.  They do not recommend going above this - but
> as I recall the maximum filesystem allocation can be huge.
> In the XFS in Iris you can set allocation units to be 1MB long [or
> more].  Which is ideal for large media files to keep the files
> contiguous on the disk sub-system.

It can be set.  The problem isn't that there -are- different block sizes,
the problem is -detecting- them without resorting to using 'ls'.

> I can see the problem with getting it from multiple sources.
> But why not look at the lenght of the file as it grows and look at
> it in bytes then it should be portable anywhere.   As far as I know
> the byte size will be indentical - and that's probably all you can
> count on.

Because it doesn't grow in bytes.  The st_size value remains a constant
from the file's creation--the maximum size that the file will ever be.  It
allocates the entire file so that it can seek to a segment start point and
write the appropriate segment from any given individual source (multiples
are done at once).

Byte sizes are useless.  You have to use blocks for this particular task,
AFAICT.

> You mean so that an 'ls' will show the size of the file, but
> the 'df' will show only the blocks that have been used.
> 
> It's been years since I've diddle with these but an lseek will
> let you extend a file beyond the current EOF and as you add
> data the 'holes' in the middle will be filled.

Exactly what I meant, and thanks.  :)  The man page says you're correct.
Appreciate that tidbit!

And that's what they're doing in cases like this--extending the
file out to the full length, then filling in the gaps as each segment
becomes available.

mark->
-- 
Bring the web-enabling power of OneGate to -your- filePro applications today!

Try the live filePro-based, OneGate-enabled demo at the following URL:
               http://www2.onnik.com/~fairlite/flfssindex.html