How should filePro-based updates be done with live-data, and live-programs involved?

[Boaz Bezborodko]

>Date: Wed, 4 Apr 2007 08:39:05 -0400
>From: "John Esak" <john at valar.com>
>Subject: How should filePro-based updates be done with live-data and
>	live-programs involved?
>To: "Filepro-List at Lists. Celestial. Com"
>	<filepro-list at lists.celestial.com>
>Message-ID: <FEEGKIBANIDBFHOPNKOLCEGCCGAA.john at valar.com>
>Content-Type: text/plain;	charset="iso-8859-1"
>
>
>Hello all,
>
>I would like to pose a simple question for discussion. It has lots of
>ramifications though, and no immediately visible simple answer. People who
>have bought (and are buying) the Survivor Accounting package are going to
>have a discussion about this some time soon in the FP Room, but I mentioned
>to them that I would put a thread up here (and the filePro Forum) to get
>things started and get some ideas from those who don't have the program and
>won't be buying it but still have thoughts on the subject.
>
>"What is the best way for me to release update, upgrades (i.e., new
>versions) of the Survivor Accounting CD?"
>
>The problem is this. The accounting system I'm providing is NOT a binary
>program. I can't simply put out Version 2.0 and say put this over the top of
>what you already have. Since the system is written entirely in filePro and
>the source code is provided so that it can be modified to suit individual
>needs, the standard way of releasing new versions is impossible. Not only
>can people change/modify the source code (processing tables, maps, screens,
>etc.), but they obviously *use* the programs so their data is unique with
>its own key files, indexes, selsets, etc.
>
>So, if I release a CD with new code in the processing tables, or new/changed
>field in the map, how can it be done so no one accidentally destroys their
>live/working system?
>
>A couple of basic parameters for the discussion that are immutable, are as
>follows. The CD must be usable a "new" install for people who are buying the
>program from scratch. In other words, they are buying in at Version 2.7 and
>they have never had the programs working before. No problem for them. But,
>the CD has to have its keys and indexes ready to go. Actually, I arrange it
>so the critical configuration key is filled on-the-fly when you begin
>configuring the system, and all the other keys are empty. However, the
>indexes (which also are created on the fly) do have to have a pre-determined
>size and shape.
>
>In the past, on other systems, I've just released updates with no keys, not
>even a zero length one (that would be disastrous). However, I was able to
>assume (I hate that word) that the maps would not have been changed by the
>user. With the accounting, I can not be sure of that at all. In fact, I
>would say most people do (will) adjust some of the maps to their taste.
>(Hopefully, they won't change crucial keys with indexes and so on, but you
>never know). If they are competent they can certainly change anything they
>want.
>
>How do I put the upgrades out then? Changing the key to something like
>newkey and the indexes to newindex.A, newindex.B... and then providing a
>script that moves them into place on demand has some merit, but which ones
>would the script move? Ask a question about each file?  That's odious.
>
>Maybe put two completely different installs on the CD, one for brand new
>users that goes on complete and as it does now... and one which is for
>people who already have the system up and running. This install would have
>*no* keys and *no indexes. Not bad... but, what about maps? Have a "newmap"
>also?, and only move it into place if requested? There are lots of maps...
>this is also pretty onerous if not odious.
>
>Anyway, you are getting the idea. In the 1.1.0 CD I'm sending out tomorrow
>(yes, we jumped right over 1.0.2 :-)) I do not have anything other than the
>full new version CD... and NO protection of any kind if someone installs it
>over their old system. If they do, anything they had will be replaced.  I'm
>just going to issue brightly colored and HUGE warnings about destroying your
>live data... and suggesting that people copy the things they want very
>judiciously. It's the only way I can think of at present, and indeed, it
>just may be the only sane thing to do.  However, I feel I would like to
>provide something foolproof, and I'm up for doing a reasonable amount of
>work on it...  What I don't want is something SO work-intensive for the
>person doing the upgrading. I really don't want to show them each map, each
>processing table... just to see if it should be upgraded or not. Yes, I can
>test old formats against new and know which have been changed.... but this
>means I have to keep all the previous levels on the CD as well as the new
>ones... and it is just too cumbersome an idea to implement.
>
>Kinda makes me wonder why I didn't just write some "binary" thing everybody
>wants and not get myself into such situations. :-)  But I've been wondering
>this for my whole life... and some things just won't ever change. Besides,
>the completely generic accounting base in filePro is such a good thing and
>has been needed for so long, somebody had to do it.
>
>Let's have at it... all good ideas welcome of course. Obviously, this is
>about my product, but it is a core problem for filePro-based products, so I
>feel it belongs in open non-OT discussion. I would appreciate it SO much if
>we can do this one with just our brains... and no flames... not even any
>flickers. If you're not interested, just press D. Thanks.
>
>John
>
>P.S. By the way, I did put up (in a private email note to owners of the
>accounting package) a tentative date/time for this discussion in the FP room
>of either 9am Wednesday, or 6pm Wednesday (EDT) next week. I'll post up
>whatever is agreed upon as it happens.
>
>  
>
I think that this can be done so long as some things are kept in mind.  
The functionality of the G/L package is the primary purpose of the 
system, so it's functionality must be kept intact.  The 2 other primary 
packages are A/P and A/R.  (IMO, Payroll is too difficult to keep 
maintained properly because of the constant changing nature of the tax 
code etc.)  Rules should be established as to what the original package 
will not touch and how changes will be made.  If you keep to those rules 
then you'll have an easy migration path, if not then you're on your own.

You would have to give some detailed information on what is being 
changed between each versions so that each programmer can go back and 
review their own programs to see what needs to be changed and what can 
be left alone.  In fact, anyone making changes should document these 
changes so that they know what they did differently.

I've only now been going over the code to see what it is I need to do to 
integrate my system to yours and I am trying to be mindful of how to do 
it while touching as little of the original code as possible and not 
touching the maps except by adding additional fields in a manner similar 
to what you described.  I've already decided that I will be using your 
A/P system and most of your A/R system almost intact.

There is one place where FPtech could really help us that we've 
discussed in the past.  The ability to keep have an entry field on the 
screen that is shorter than the field length would allow you to 
manipulate the size of field lengths without having to make any changes 
to the screen itself.  (The data could scroll within the entry window 
similar to how it does in the Printer Maintenance screen.)  There are 
probably some issues regarding displaying the data or putting it on a 
report that I haven't fully taken into account, but it means that the 
general structure of the programs and code wouldn't be affected all that 
much.

The most important thing is to keep in mind that any changes must be 
looked upon as building "on top of" instead of "over" the existing data 
and calls to make upgrading as easy as possible.  So new code should be 
in CALL tables if possible or in subroutines so as to make them easy to 
distinguish.

And if the changes have to be made manually by the programmer, well, 
that their responsibility.  There is only so much that you can be 
expected to take care of especially given the cost of the package.

Boaz