testing for corruption

[Fairlight]

At Mon, Feb 04, 2008 at 05:27:01PM -0800 or thereabouts, 
suspect Jeff Harrison was observed uttering:
> This is all most people care about isn't it?  I would
> want to keep this very simple at its basic level, then
> people could then use flags for options to do the more
> detailed validity checking.

Not really?  Well...okay, it's not all -I- would care about, but as my
admins at my ISP would note, I'm not "most people" in terms of what I want.

> > Further...when you find such a misalignment, what is
> > the procedure for
> > continuing reporting?  Reset the record start offset
> > to where you -think-
> > the record actually starts?  Continue onwards as if
> > nothing happened,
> > assuming the other offsets should be correct? 
> 
> Yes.  Continue on.  Lets not make this too complex
> -yet.  There would need to be another utility to
> actually fix the problem(s) - or you just need to fix
> it the old fashoned way - then run this utility again
> to see if you are clean.

It's not a matter of fixing the problem.  Which, honestly, I don't consider
this kind of problem automation-friendly for fixing.  That's asking for
trouble when detection alone is not sure-fire.

Why make it more complex?  Because if you truly had a HD dying and a
segment from the middle of a file got dropped, your offsets very probably
-would- be off.  It's a more serious corruption than what you were trying
to detect, and you're (apparently) saying it's too complex to consider.
That's paradoxical.

> I don't care what the cause the of corruption is - I
> keep seeing people ask this list how they can check
> for invalid data in fields - this would fit the bill
> for that, I think.

That's a mistake--not caring why it was corrupted.  Forensically, that is a
grave tactical error; ignoring the root cause leaves you a high probability
of losing yet more data if you don't address the root cause and it is
indeed failing hardware that's at the root of it.

If you care enough to know whether valid data is in the real fields, I
should think caring about the records even being aligned correctly, and the
implication of imminent hardware failure would be even higher on your list
of priorities.  What good is knowing if your brake fluid is at precisely
the right level when the transmission is dying and about to fall onto the
ground?  

The fact that there are two theoretical types of corruption, high (app) and
low (filesystem) level, means that this can't be "simply" a check for one
without allowing for the possibility of the other, no matter how rare it
should be.  Not if it's to be done correctly.

mark->