pseudo-RSS initial example/spec

[Fairlight]

Hi all,

I've given this a bit of thought, and I've come up with an initial proposal
based on what I'd mentioned earlier about my thoughts on how RSS could be
integrated with filePro.  I've tossed the actual RSS file specification
itself, while accomodating the important parts of the model.

Some of what I'm proposing is subject to availability of features.  If
variable lookup targets, indices, and qualifiers are available, a generic
processing module is certainly ideal.

If such facilities are not available, then the facility would have to
be purpose-built in some respects, depending on the processing.  The
table, index, key, key_value, and qualifier could serve as triggers for
multi-conditional lookup initialisations, whereby only one was triggered,
based on the data presented.  This could be clumsy in complicated
processing scenarios, but not too bad in more modest scenarios.  It simply
depends on how much is done.  A better alternative in this event may be to
only allow one feed to affect one table, period.

What follows is what I would initially propose as a DTD for doing a
pseudo-RSS feed for data synchronisation between systems with filePro and
an httpd server onboard.  Following the DTD is an example of a short feed
file, with comments.  The last section is an explanation of how the entire
process should work.

Comments are welcome.

mark->



<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE fl_fprss [
<!ELEMENT fl_fprss (generator,version,publish_date,publish_time,host,record*)>
<!ELEMENT generator (#PCDATA)>
<!ELEMENT version (#PCDATA)>
<!ELEMENT publish_date (#PCDATA)>
<!ELEMENT publish_time (#PCDATA)>
<!ELEMENT host (#PCDATA)>
<!ELEMENT record (record_publish_date,record_publish_time,field*)>
<!ATTLIST record type (add|update|delete) "add">
<!ATTLIST record table CDATA #IMPLIED>
<!ATTLIST record index CDATA #IMPLIED>
<!ATTLIST record key CDATA #IMPLIED>
<!ATTLIST record key_value CDATA #IMPLIED>
<!ATTLIST record qualifier CDATA #IMPLIED>
<!ELEMENT record_publish_date (#PCDATA)>
<!ELEMENT record_publish_time (#PCDATA)>
<!ELEMENT field (field_number,field_value)>
<!ELEMENT field_number (#PCDATA)>
<!ELEMENT field_value (#PCDATA)>
]>
<!-- Generator Format:  Program Name Here -->
<!--   Version Format:  v00.00.01         -->
<!--      Date Format:  MM/DD/YYYY        -->
<!--      Time Format:  HH:MM:SS          -->
<!--      Host Format:  Arbitrary String  -->

<!-- The 'host' field is used to denote which machine the request actually
     came from.  This, combined with the date and time, creates a unique
     'ID' that can be used to validate whether the record has already been
     processed from a previous retrieval, or whether it needs to be
     handled for the first time.
-->

<!-- FL_FPRSS EXAMPLE. -->
<fl_fprss>

<!-- EXAMPLE OF FEED FILE HEADERS -->
     <generator>fl_fprss</generator>
     <version>v00.00.01</version>
     <publish_date>07/19/2004</publish_date>
     <publish_time>13:42:04</publish_time>
     <host>Web Server 1</host>
<!-- END EXAMPLE OF FEED FILE HEADERS -->

<!-- EXAMPLE:  Add Records. -->
     <record type="add" table="users">
          <record_publish_date>07/19/2004</record_publish_date>
          <record_publish_time>12:42:02</record_publish_time>
          <field>
               <field_number>1</field_number>
               <field_value>Fairlight</field_value>
          </field>
          <field>
               <field_number>2</field_number>
               <field_value>Admin</field_value>
          </field>
     </record>
     <record type="add" table="users">
          <record_publish_date>07/19/2004</record_publish_date>
          <record_publish_time>12:40:34</record_publish_time>
          <field>
               <field_number>1</field_number>
               <field_value>Jim</field_value>
          </field>
          <field>
               <field_number>2</field_number>
               <field_value>User</field_value>
          </field>
     </record>
<!-- END EXAMPLE:  Add Records. -->

<!-- EXAMPLE:  Update Records. -->
     <record type="update" table="account" key="B" key_value="Luljak">
          <record_publish_date>07/19/2004</record_publish_date>
          <record_publish_time>11:50:34</record_publish_time>
          <field>
               <field_number>3</field_number>
               <field_value>39.95</field_value>
          </field>
     </record>
<!-- END EXAMPLE:  Update Records. -->

<!-- EXAMPLE:  Delete Records. -->
     <record type="delete" table="transactions" key="A" key_value="L42670023">
          <record_publish_date>07/19/2004</record_publish_date>
          <record_publish_time>10:30:12</record_publish_time>
          <field>
               <field_number />
               <field_value />
          </field>
     </record>
<!-- END EXAMPLE:  Delete Records. -->

</fl_fprss>
     
<!-- END OF FL_FPRSS EXAMPLE. -->


<!-- EXPLANATION:

First step is to have filePro SYSTEM() off a fetching mechanism (wget,
curl, lynx, etc.) to retrieve the file from the system at hand.  There may
be more than one, so everything below this point is repeated for each host.

The parser (whatever processing form that may take--I'm still waiting to
see if tables, qualifiers, and indexes can be variables, or if this has
to be special code for each application) should check the host, publish
date, and publish time of the entire feed file itself to see if it has been
processed before.  (This is information that should be tracked at each
run of the pseudo-RSS process.)  If they have, there is not a new table
available, and we should simply stop processing at this point so as not to
waste time.  Possibly we log the fact that no update was available, for
debugging/troubleshooting purposes.

Assuming we need to act on a new feed file, the code should ostensibly look
through every record request in the log, determine if the uniqueness of
the host, publish date, and publish time of the record request is actually
representative of a new request, or if it has been handled already.  There
should be request tracking in this regard.

If we determine that the record request has already been processed in the
past, we disregard the request.

If we determine that the record request is a brand new request, we should
effect the requested changes and track the request as having been
processed.

END OF EXPLANATION -->