Re cloud server(Fairlight)
Fairlight
fairlite at fairlite.com
Fri Mar 18 16:47:23 PDT 2016
This would have made some sort of sense if what you were saying didn't
violate a core principle of networking.
https://en.wikipedia.org/wiki/Maximum_transmission_unit
Windows can't do a 64K packet, as the largest allowed MTU size is 1500
bytes. And in practise, I've actually seen smaller MTUs (1436, for
instance, in a PPPoE over DSL scenario). Even if you enable jumbo frames,
the best you're getting out of it is 9000 bytes per packet, and that's
likely only valid inside a LAN.
Anything over the MTU size is -automatically- fragmented into multiple
packets. This was beneficial in analog modem days, as you could specify a
smaller MTU, and thus ensure that smaller packets for things like single
characters in telnet got a bit better turn at the trough within the stack,
compared to a solidly filled FTP data connection.
Now, you -can- adjust NFS to something like an 8KB size, which makes sense
if you have jumbo frames for the entire path. There's zero sense in trying
to send 64K over a single packet, though; I'm not seeing anything which
supports an MTU that high.
Also, bear in mind the packet header takes up part of the maximum packet
size.
In any event, Windows has to play by the same rules as everyone else with a
standard TCP/IP stack. Issue this at a command prompt:
netsh interface ipv4 show subinterfaces
I'm on Windows 7 Professional 64-bit, and I've got a giant size only on
loopback. Bluetooth is 1477. Everything else (including my VMWare,
ethernet, and wireless connections are all 1500.
So honestly, I don't know what the hell you're talking about with the
64K nonsense. Sure, they can try to send it, but the TCP/IP stack will
automagically fragment a packet that size into 45 normal packets, given an
MTU of 1500, and the TCP header size of 20 bytes. 65536 / 1480 is 44.281,
so 45 packets.
The only time you really run into problems with small packets is in
contention against larger packets, which will take more time on a smaller
pipe (read: analog modem speeds). Packet size hasn't been a valid limiting
factor since we moved past analog modems, honestly.
I've heard of adjusting window sizes, but I don't think that has anything
to do with bypassing the MTU. I could be wrong; I'm not a dedicated
network engineer. But I've never seen an MTU change dynamically on an
interface under -any- OS. I suspect whatever is scaling would scale
application-level packets to multiples of the packet data size so that you
don't fragment and inject 'air' into your last TCP/IP packet.
At least part of what you said is incorrect, according to everything I've
learned, combined with an actual -look- at the Windows TCP/IP stack
settings on a production system.
mark->
On Fri, Mar 18, 2016 at 03:34:39PM +0000, Jason Garner via Filepro-list thus spoke:
> Interesting conversation you guys are having.
>
> Our FilePro applications here have anywhere from 40 to 70mbps bursts on a
> gigabit pipe.
>
> The problem with Windows and smb file transfers is that it does not like
> lots of small data transfers over the network. Linux has always been better
> at handling network data transmissions and multicast streaming stuff
> because it is very good at automatically adjusting the TCP transmit and
> receive side window scaling.
>
> Windows takes a very Henry Ford type of approach to this. You can transmit
> any size data chunk you like, so long as its transmitted in a 64k payload.
>
> Filepro transmissions are small..1k 4k etc. So in that 40 to 70mbps, you
> are only really sending probably 10mbps. The rest of it is air.
>
> So I almost kind of wonder if you had a 100/10 cable connection if you
> would notice much of a difference.
>
> There is also 3rd party software you can install that does compression and
> also auto adjusts the TCP windows on the fly. It might be doable, who knows.
>
> On Fri, Mar 18, 2016 at 5:33 AM <filepro-list-request at lists.celestial.com>
> wrote:
>
> > Send Filepro-list mailing list submissions to
> > filepro-list at lists.celestial.com
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > http://mailman.celestial.com/mailman/listinfo/filepro-list
> > or, via email, send a message with subject or body 'help' to
> > filepro-list-request at lists.celestial.com
> >
> > You can reach the person managing the list at
> > filepro-list-owner at lists.celestial.com
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Filepro-list digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Re: cloud server (Fairlight)
> > 2. Re: cloud server (Jose Lerebours)
> > 3. Re: cloud server (Fairlight)
> > 4. Re: cloud server (Fairlight)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Thu, 17 Mar 2016 19:22:25 -0400
> > From: Fairlight <fairlite at fairlite.com>
> > To: filepro-list at lists.celestial.com
> > Subject: Re: cloud server
> > Message-ID: <20160317232225.GY24133 at iglou.com>
> > Content-Type: text/plain; charset=iso-8859-1
> >
> > Are you actually intending to use it as realtime storage, or backup for the
> > application?
> >
> > If realtime, you're nuts. If you think Windows clients perform poorly on a
> > LAN when they have to drag the whole index over to look something up, wait
> > until you slow it down to only broadband speeds, unless you're on something
> > like Google Fiber or maybe FIOS.
> >
> > If backup, that's another story altogether.
> >
> > m->
> >
> > On Thu, Mar 17, 2016 at 06:28:27PM -0400, Jose Lerebours via Filepro-list
> > thus spoke:
> > > 250GB or better.
> > > filePro application.
> > > price, well, as affordable as possible.
> > >
> > > 250GB or 20GB makes no much of a difference given that it will be
> > > for a lightweight filePro application.
> > >
> > > thanks,
> > >
> > >
> > >
> > >
> > > On 03/17/2016 06:24 PM, Fairlight via Filepro-list wrote:
> > > >On Thu, Mar 17, 2016 at 06:08:10PM -0400, Jose Lerebours via
> > Filepro-list thus spoke:
> > > >>Who do you guys use or recommend out there?
> > > >>
> > > >>preferably unix but will settle for linux.
> > > >For what kind of use? Documents? Whole system backups? Something
> > else?
> > > >
> > > >How much space do you need?
> > > >
> > > >What price range is acceptable?
> > > >
> > > >NEI ? Not Enough Information
> > > >
> > > >m->
> > >
> > > _______________________________________________
> > > Filepro-list mailing list
> > > Filepro-list at lists.celestial.com
> > > Subscribe/Unsubscribe/Subscription Changes
> > > http://mailman.celestial.com/mailman/listinfo/filepro-list
> > >
> >
> > --
> > Audio panton, cogito singularis.
> >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Thu, 17 Mar 2016 19:47:17 -0400
> > From: Jose Lerebours <fpgroups at gmail.com>
> > To: filepro-list at lists.celestial.com
> > Subject: Re: cloud server
> > Message-ID: <56EB4205.7060203 at gmail.com>
> > Content-Type: text/plain; charset=windows-1252; format=flowed
> >
> > I must confess, this is totally new to me. It blows my mind how is it
> > that today we talk about terrabites, gigabites and 10,000mbps up/down
> > etc. and still look at it as "slow".
> >
> > In the not so far back times, xenix with 28k modem and a 10mb hd ran an
> > outfit with nothing more than a couple of multiplexers to handle all the
> > serial lines between locations.
> >
> > You mention "windows clients" - Is the drag caused by the client or the
> > application? I really never heard of filePro to be a drag or hog for
> > bandwidth.
> >
> > I appreciate the educational info!
> >
> > Thanks,
> >
> >
> >
> >
> > On 03/17/2016 07:22 PM, Fairlight via Filepro-list wrote:
> > > Are you actually intending to use it as realtime storage, or backup for
> > the
> > > application?
> > >
> > > If realtime, you're nuts. If you think Windows clients perform poorly
> > on a
> > > LAN when they have to drag the whole index over to look something up,
> > wait
> > > until you slow it down to only broadband speeds, unless you're on
> > something
> > > like Google Fiber or maybe FIOS.
> > >
> > > If backup, that's another story altogether.
> > >
> > > m->
> > >
> > > On Thu, Mar 17, 2016 at 06:28:27PM -0400, Jose Lerebours via
> > Filepro-list thus spoke:
> > >> 250GB or better.
> > >> filePro application.
> > >> price, well, as affordable as possible.
> > >>
> > >> 250GB or 20GB makes no much of a difference given that it will be
> > >> for a lightweight filePro application.
> > >>
> > >> thanks,
> > >>
> > >>
> > >>
> > >>
> > >> On 03/17/2016 06:24 PM, Fairlight via Filepro-list wrote:
> > >>> On Thu, Mar 17, 2016 at 06:08:10PM -0400, Jose Lerebours via
> > Filepro-list thus spoke:
> > >>>> Who do you guys use or recommend out there?
> > >>>>
> > >>>> preferably unix but will settle for linux.
> > >>> For what kind of use? Documents? Whole system backups? Something
> > else?
> > >>>
> > >>> How much space do you need?
> > >>>
> > >>> What price range is acceptable?
> > >>>
> > >>> NEI ? Not Enough Information
> > >>>
> > >>> m->
> > >> _______________________________________________
> > >> Filepro-list mailing list
> > >> Filepro-list at lists.celestial.com
> > >> Subscribe/Unsubscribe/Subscription Changes
> > >> http://mailman.celestial.com/mailman/listinfo/filepro-list
> > >>
> >
> >
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Thu, 17 Mar 2016 20:32:12 -0400
> > From: Fairlight <fairlite at fairlite.com>
> > To: filepro-list at lists.celestial.com
> > Subject: Re: cloud server
> > Message-ID: <20160318003212.GZ24133 at iglou.com>
> > Content-Type: text/plain; charset=iso-8859-1
> >
> > On Thu, Mar 17, 2016 at 07:47:17PM -0400, Jose Lerebours via Filepro-list
> > thus spoke:
> > > I must confess, this is totally new to me. It blows my mind how is
> > > it that today we talk about terrabites, gigabites and 10,000mbps
> > > up/down etc. and still look at it as "slow".
> > >
> > > In the not so far back times, xenix with 28k modem and a 10mb hd ran
> > > an outfit with nothing more than a couple of multiplexers to handle
> > > all the serial lines between locations.
> > >
> > > You mention "windows clients" - Is the drag caused by the client or
> > > the application? I really never heard of filePro to be a drag or
> > > hog for bandwidth.
> > >
> > > I appreciate the educational info!
> >
> > The problem is that storage sizes are increasing simultaneously with
> > transmission speeds.
> >
> > Sure, I used to have only 500GB of storage and a 1.5mbit down, 256kbit up
> > ADSL circuit. Now I have 62mbit down, 6.2mbit up cable broadband, but I
> > also have 6TB of storage space to back up. Not all of that is in use, but
> > it's my capacity.
> >
> > With compression, it takes 4.5 days to do a full backup and validation
> > with Acronis TrueImage to a WD EX4 NAS on my WiFi LAN. That's actually
> > the recommended idiom for the EX4, is putting it directly on your router.
> > I had it directly hooked to the 1GB ethernet on my system at first, and
> > I saw -no- speed improvement at all. It's the WD EX4's CPU, which is
> > under-specced for the throughput of anything higher than 50mbit.
> >
> > As I was saying, it takes 4.5 days ? to back up only 2.46TB of data in a
> > LAN situation.
> >
> > Backup tech has not kept pace with the growing capacity of
> > consumer-oriented, commoditised storage. To get anything fast -and- large,
> > you need to spend thousands. That EX4 was $600. I remember looking at
> > other solutions, "real" NAS vendors, SAN, etc. It is -not- affordable
> > unless you have a huge revenue stream to justify getting into it.
> >
> > In general, a lot of things have fallen by the wayside during this period
> > of tech growth since 1993. Games used to have tightly written netcode
> > which could work over dialup, and now developers struggle to get
> > multiplayer working properly with a lot lower latency connections on
> > broadband. A whole Linux -distribution- used to be able to run on a 4MB
> > 386sx/25 (including TinyX!), and now the -kernel- won't even fit into 4MB,
> > let alone applications. The linux kernel started requiring more RAM at
> > version 2.4. I know, as the Cobalt Qube I had required a firmware flash to
> > let it take anything newer than 2.2. So on the whole, tech is getting to
> > be less expensive, we have more capacity, higher speeds, and...fallen
> > standards. That's a bad scenario any day of the week.
> >
> > As for the Windows/filePro/networking/speed issue, you have to understand
> > what's happening under the hood. The same thing holds true on Windows and
> > in *nix over NFS, by the way.
> >
> > Basically, think about your key or data segment. I really only ever use
> > key, so we'll standardise on that for the scope of this discussion. If you
> > look at your key, it's the full set of data. If you're working with
> > relational tables, it's more than one key file, as well. Let's say those
> > are the 32-bit limit of 2GB in size, just for the sake of argument. Now,
> > you have indexes. Those are much smaller files which contain information
> > which keeps you from having to sequentially read up to 2GB worth of data
> > every time you want something. But if you reference the spec for indexes
> > at http://fptech.com/Products/Docs/FPFormat/autoix45.shtml you'll see the
> > math behind the BTree+ indexes. You can figure out what fields you've used
> > for an index, do the math, and extrapolate out to the index size for a 2GB
> > file.
> >
> > Every time you look something up, the index is read. I'm not sure if the
> > whole index needs to be read, or if BTree+ lets you stop early in the tree
> > once you find what you're looking for. It really probably depends on the
> > purpose. If you're running a report, it's probably more likely to need the
> > whole thing than if you're searching for a specific record. But assume you
> > need the whole index. Where's it stored? It's stored either on the
> > Windows server hosting the files, or it's stored on a NAS/SAN for *nix,
> > usually under NFS. In Windows, your client applications must drag it over
> > the network to read it, since there's no local copy. This isn't like SQL,
> > where you submit a query and the server handles the heavy lifting. The
> > filePro binaries on the client run identically to the ones on the server,
> > meaning they must read the whole file ? except the clients must read it
> > across the network. That entails IP, TCP, -and- protocol-level network
> > overhead. The same is true of *nix systems using NFS or CIFS for their
> > storage. And I can tell you that NFS was originally designed -so- bloated
> > that it was never meant to run on under 10mbit, just due to the overhead.
> > It is -not- an elegant protocol. It has probably improved by v4, but
> > still, it's not lightweight.
> >
> > This is the difference between client-server architectures:
> >
> > * With SQL the server takes a tiny query, does the heavy lifting directly
> > on the server, and spits out -only- the result. All file manipulation
> > is in local storage (unless a SAN is involved, which is definitely
> > possible in large deployment scenarios, but it's still less of an issue
> > due to the way the engines are optimised, and the faster protocols
> > often in play (iSCSI being one example). Also, you usually design
> > the servers with decent provisioning and enough RAM to cache decently.
> >
> > * With filePro, the server is simply a datastore. The 'client' is really
> > written identically to the server, and requires that the index be read
> > (possibly fully) in order to obtain the address within the key file to
> > seek to to read the -actual- data. Then it has to actually seek there
> > and grab the data from the key file. I can't speak to CIFS, but NFSv2
> > and higher support and perform seek() on the server-side, so at least
> > it doesn't have to ship the key file up to the seek point plus record
> > size. That said, if you are running a report which has to access a lot
> > of (or all!) records, you -will- be shipping the majority or entirety
> > of the key file over as well, since the non-index data is not stored in
> > the index.
> >
> > And this is why the fP client/server "model" sucks, compared to SQL. Well,
> > one amongst several reasons. We could get into the whole ACL issue, but
> > that's outside the scope of this discussion. There are others, as well.
> >
> > Doing this processing with large files on gigabit LAN is one thing,
> > although -still- painful, depending on the elegance/efficiency of the
> > coding, and the job requirements. Doing it over mere broadband which
> > crosses even one hop which is narrower? That's going to be far less
> > pleasant. Also, you need to take fault tolerance (or lack thereof) into
> > account. Across a WAN, iSCSI is a Very Bad Idea[tm]. I've hit fault
> > tolerance issues with it on a LAN. I don't want to think about it on a
> > WAN. An edge router falls over for some reason (RAM exhaused, BGP table
> > falls on the floor, name it...), and you've suddenly got a risk of data
> > corruption.
> >
> > Oh, and compounding the issue, I'm guessing a lot of filePro "thin client"
> > boxes are built with very little RAM. That's going to make disk caching
> > next to non-existent, which means you can't even count on caching for
> > performance benefits.
> >
> > I would strongly recommend rethinking using cloud storage in realtime with
> > filePro.
> >
> > mark->
> > --
> > Audio panton, cogito singularis.
> >
> >
> > ------------------------------
> >
> > Message: 4
> > Date: Thu, 17 Mar 2016 20:34:40 -0400
> > From: Fairlight <fairlite at fairlite.com>
> > To: filepro-list at lists.celestial.com
> > Subject: Re: cloud server
> > Message-ID: <20160318003440.GA24133 at iglou.com>
> > Content-Type: text/plain; charset=iso-8859-1
> >
> > He may be. He wasn't explicit.
> >
> > If it's a "terminal session only" scenario to a VPS or other VM where fP is
> > hosted and run natively on the same storage as the data, okay, that may be
> > fine.
> >
> > If it's a situation where the data itself is stored somewhere in the cloud,
> > and a share from Timbuktu is mounted to some internal Linux system, you're
> > talking about a nightmare.
> >
> > mark->
> >
> > On Thu, Mar 17, 2016 at 08:08:29PM -0400, Scott Walker thus spoke:
> > > Mark,
> > >
> > > I think Jose is talking about using Linux. This is not going to be
> > Windows
> > > filepro, but fp running on Linux. Only something like Anziowin will be
> > > running on the client side, emulating a terminal. No speed issue running
> > > over the internet.
> > >
> > > I have customers do it all day long. Response time is good.
> > >
> > > Sorry to reply directly to you guys but my posts to the list are bouncing
> > > again.
> > >
> > > Regards,
> > > Scott
> > >
> > >
> > > -----Original Message-----
> > > From: Filepro-list
> > > [mailto:filepro-list-bounces+scottwalker
> > =ramsystemscorp.com at lists.celestial.
> > > com] On Behalf Of Jose Lerebours via Filepro-list
> > > Sent: Thursday, March 17, 2016 7:47 PM
> > > To: filepro-list at lists.celestial.com
> > > Subject: Re: cloud server
> > >
> > > I must confess, this is totally new to me. It blows my mind how is it
> > that
> > > today we talk about terrabites, gigabites and 10,000mbps up/down etc. and
> > > still look at it as "slow".
> > >
> > > In the not so far back times, xenix with 28k modem and a 10mb hd ran an
> > > outfit with nothing more than a couple of multiplexers to handle all the
> > > serial lines between locations.
> > >
> > > You mention "windows clients" - Is the drag caused by the client or the
> > > application? I really never heard of filePro to be a drag or hog for
> > > bandwidth.
> > >
> > > I appreciate the educational info!
> > >
> > > Thanks,
> > >
> > >
> > >
> > >
> > > On 03/17/2016 07:22 PM, Fairlight via Filepro-list wrote:
> > > > Are you actually intending to use it as realtime storage, or backup
> > > > for the application?
> > > >
> > > > If realtime, you're nuts. If you think Windows clients perform poorly
> > > > on a LAN when they have to drag the whole index over to look something
> > > > up, wait until you slow it down to only broadband speeds, unless
> > > > you're on something like Google Fiber or maybe FIOS.
> > > >
> > > > If backup, that's another story altogether.
> > > >
> > > > m->
> > > >
> > > > On Thu, Mar 17, 2016 at 06:28:27PM -0400, Jose Lerebours via
> > Filepro-list
> > > thus spoke:
> > > >> 250GB or better.
> > > >> filePro application.
> > > >> price, well, as affordable as possible.
> > > >>
> > > >> 250GB or 20GB makes no much of a difference given that it will be for
> > > >> a lightweight filePro application.
> > > >>
> > > >> thanks,
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On 03/17/2016 06:24 PM, Fairlight via Filepro-list wrote:
> > > >>> On Thu, Mar 17, 2016 at 06:08:10PM -0400, Jose Lerebours via
> > > Filepro-list thus spoke:
> > > >>>> Who do you guys use or recommend out there?
> > > >>>>
> > > >>>> preferably unix but will settle for linux.
> > > >>> For what kind of use? Documents? Whole system backups? Something
> > > else?
> > > >>>
> > > >>> How much space do you need?
> > > >>>
> > > >>> What price range is acceptable?
> > > >>>
> > > >>> NEI ? Not Enough Information
> > > >>>
> > > >>> m->
> > > >> _______________________________________________
> > > >> Filepro-list mailing list
> > > >> Filepro-list at lists.celestial.com
> > > >> Subscribe/Unsubscribe/Subscription Changes
> > > >> http://mailman.celestial.com/mailman/listinfo/filepro-list
> > > >>
> > >
> > > _______________________________________________
> > > Filepro-list mailing list
> > > Filepro-list at lists.celestial.com
> > > Subscribe/Unsubscribe/Subscription Changes
> > > http://mailman.celestial.com/mailman/listinfo/filepro-list
> > >
> > >
> >
> > --
> > Audio panton, cogito singularis.
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > Filepro-list mailing list
> > Filepro-list at lists.celestial.com
> > http://mailman.celestial.com/mailman/listinfo/filepro-list
> >
> >
> > ------------------------------
> >
> > End of Filepro-list Digest, Vol 146, Issue 15
> > *********************************************
> >
> --
> Jason Garner
> Systems Administrator
> 1801 Oberlin Rd, Suite 204
> Middletown, PA 17057
> Work 717-985-1122 x 1139
> Mobile 717-645-3521
> jason.garner at evalsvs.com
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mailman.celestial.com/pipermail/filepro-list/attachments/20160318/be4de8f3/attachment.html>
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> Subscribe/Unsubscribe/Subscription Changes
> http://mailman.celestial.com/mailman/listinfo/filepro-list
>
--
Audio panton, cogito singularis.
More information about the Filepro-list
mailing list