raid vs network performance

3dO wrote:
> ok, this is a long one but i really am clueless on this one
>
> two week ago i set up a "fileserver", as i'm a visual nut and not a
> server nut i went for a simple xp pro machine with a raidcontroller:
> asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
> raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
> spinpoint t166 160GB as systemdisk.
>
> now when i boot up the system the disks make a loud click right after
> spinup, not having any experience with samsungs, i don't know if this is
> normal behaviour for these kind of disks, it sounds scary when you
> always worked with seagates. (they're very silent after this)
>
> the raid 5 setup on the promise card -which is pci with a default and
> unchangeable stripe size of 16KB - is the real problem
>
> when i work with a 4.01GB video dvd image i get the following
> transferrates:
>
> localWRITE: copy from systemdisk to raid5 array : 28.9MB/s sustained
> localREAD : copy from raid5 array to systemdisk : 49.4MB/s sustained
> netWRITE: copy from network to raid5 array : 14.7MB/s
> netREAD : copy from raid5 array to network : 24.5MB/s
>
> local speeds are a bit acceptable but i mainly use it in network over
> gigabit, and those speeds are just not up to my expectations. they are
> not sustained but more like /\_/\_/\_/\ (graph) in bursts, with bursts
> up to 53MB/s, looks like it's getting network traffic and then stopping
> to write it to the array...
>
> now i already swithed my raidcontroller to another pci slot because i
> noticed that it shared an IRQ with the onboard gigabit network
> controller i use to connect it to my network but that didn't help
> i tried all sorts of NTFS cluster setting on the array partition but to
> no avail
> i tried a raid 10 ... no difference (write was slightly better but not
> great)
>
> so i roamed the net to find ... nothing
> i really don't know what to do, i have the feeling that there is some
> form of issue with sharing pci resources between the pci raid controller
> and the onboard networkcard so i went back to the shop and they were
> willing to take the promise raid controller back because i was thinking
> about an areca 1210 pci express raid controller and not very pleased
> with the promise card but... i'm not completely sure that it's not
> another problem that's causing these horrible transfer rates.
>
> has anybody any idea, it would really be appreciated !
>
> thx
> 3dO

http://www.newegg.com/Product/ProductReview.aspx?Item=N82E16822152054

"Cons: After about 2 and a half weeks the drive started making a loud
clicking noise and promptly failed."

Maybe the click on your drives is significant.

Test the drives individually, on a Southbridge port, and use whatever
diagnostics Samsung offers.

TX4310 - SoftRAID, no cache
Areca 1210 - Intel IOP processor for hardware RAID, onboard cache RAM
- Download driver from Areca site, not the one on the CD.
(As posted in the Newegg reviews for the card.)
WinXP Pro - limited number of connections to "shares" (10 users) ? Not a server OS.

It may be RAID5, but it still needs backups!

Keep a matched spare disk on hand, for maintenance.

Using the matched spare disk, do a "fire drill" on the new array. Unplug
a drive, do a rebuild with the spare. Note any response from the Areca software,
like whether you get notification that the drive went missing. Practice while
nobody is using it, so you're prepared for the day when something happens
to the array. Know how many hours it takes to do the rebuild, and what
performance penalty exists while rebuilding. (Try some benchmarks on a client,
while the rebuild is underway.)

Paul
 >> Stay informed about: raid vs network performance 

3dO wrote:
> ok, this is a long one but i really am clueless on this one
>
> two week ago i set up a "fileserver", as i'm a visual nut and not a
> server nut i went for a simple xp pro machine with a raidcontroller:
> asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
> raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
> spinpoint t166 160GB as systemdisk.
>
> now when i boot up the system the disks make a loud click right after
> spinup, not having any experience with samsungs, i don't know if this is
> normal behaviour for these kind of disks, it sounds scary when you
> always worked with seagates. (they're very silent after this)
>
> the raid 5 setup on the promise card -which is pci with a default and
> unchangeable stripe size of 16KB - is the real problem
>
> when i work with a 4.01GB video dvd image i get the following
> transferrates:
>
> localWRITE: copy from systemdisk to raid5 array : 28.9MB/s sustained
> localREAD : copy from raid5 array to systemdisk : 49.4MB/s sustained
> netWRITE: copy from network to raid5 array : 14.7MB/s
> netREAD : copy from raid5 array to network : 24.5MB/s
>
> local speeds are a bit acceptable but i mainly use it in network over
> gigabit, and those speeds are just not up to my expectations. they are
> not sustained but more like /\_/\_/\_/\ (graph) in bursts, with bursts
> up to 53MB/s, looks like it's getting network traffic and then stopping
> to write it to the array...
>
> now i already swithed my raidcontroller to another pci slot because i
> noticed that it shared an IRQ with the onboard gigabit network
> controller i use to connect it to my network but that didn't help
> i tried all sorts of NTFS cluster setting on the array partition but to
> no avail
> i tried a raid 10 ... no difference (write was slightly better but not
> great)
>
> so i roamed the net to find ... nothing
> i really don't know what to do, i have the feeling that there is some
> form of issue with sharing pci resources between the pci raid controller
> and the onboard networkcard so i went back to the shop and they were
> willing to take the promise raid controller back because i was thinking
> about an areca 1210 pci express raid controller and not very pleased
> with the promise card but... i'm not completely sure that it's not
> another problem that's causing these horrible transfer rates.
>
> has anybody any idea, it would really be appreciated !
>
> thx
> 3dO

raid 5 is not for speed so much as for reliability and compactness

there can be big differences between gigabit cards

the pci bus comes in many flavors

overlay on those that no chain is stronger than its weakest link

plus -

1 GB of RAM is puny by today's standards

Microsoft makes server versions -- XP Pro isn't one of them
-- you'd probably do better with Samba running on Linux or Solaris

--
The e-mail address in our reply-to line is reversed in an attempt to
minimize spam. Our true address is of the form che....RemoveThis@prodigy.net.
 >> Stay informed about: raid vs network performance 

In comp.sys.ibm.pc.hardware.storage 3dO <noaddress.TakeThisOut@nomail.com> wrote:
> ok, this is a long one but i really am clueless on this one

> two week ago i set up a "fileserver", as i'm a visual nut and not a
> server nut i went for a simple xp pro machine with a raidcontroller:
> asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
> raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
> spinpoint t166 160GB as systemdisk.

> now when i boot up the system the disks make a loud click right after
> spinup, not having any experience with samsungs, i don't know if this is
> normal behaviour for these kind of disks, it sounds scary when you
> always worked with seagates. (they're very silent after this)

Check the SMART status. If there is nothing in there, then
the disks are fine. IF it is just the one click, I would guess
this is the head unloading and nothing to worry about.


> the raid 5 setup on the promise card -which is pci with a default and
> unchangeable stripe size of 16KB - is the real problem

> when i work with a 4.01GB video dvd image i get the following transferrates:

> localWRITE: copy from systemdisk to raid5 array : 28.9MB/s sustained
> localREAD : copy from raid5 array to systemdisk : 49.4MB/s sustained
> netWRITE: copy from network to raid5 array : 14.7MB/s
> netREAD : copy from raid5 array to network : 24.5MB/s

This controller is likely fakeRAID, i.e not a hardware controller.
For that the numbers are about right, when assuming a PCI bus.

> local speeds are a bit acceptable but i mainly use it in network over
> gigabit, and those speeds are just not up to my expectations. they are
> not sustained but more like /\_/\_/\_/\ (graph) in bursts, with bursts
> up to 53MB/s, looks like it's getting network traffic and then stopping
> to write it to the array...

> now i already swithed my raidcontroller to another pci slot because i
> noticed that it shared an IRQ with the onboard gigabit network
> controller i use to connect it to my network but that didn't help
> i tried all sorts of NTFS cluster setting on the array partition but to
> no avail
> i tried a raid 10 ... no difference (write was slightly better but not
> great)

> so i roamed the net to find ... nothing
> i really don't know what to do, i have the feeling that there is some
> form of issue with sharing pci resources between the pci raid controller
> and the onboard networkcard so i went back to the shop and they were
> willing to take the promise raid controller back because i was thinking
> about an areca 1210 pci express raid controller and not very pleased
> with the promise card but... i'm not completely sure that it's not
> another problem that's causing these horrible transfer rates.

> has anybody any idea, it would really be appreciated !

We have some Arceas in use with 8 disks as RAID 6 on
them. Read speed exceeds 200MB/s with a 64 bit/133MHz
PCI-X bus. I have not benchmarked the write speed, but
it is likely in the same class. Note that PCI cannot
gice you these speeds, it levels out at about 100MB/s
combines speed. Your localREAD speeds are very likely the
maximum the PCI bus of your machine can support.
Software RAID and fakeRAID can saturate the bus of the
machine, but it can need more bandwidth compared to
a hadrware RAID controller, which is, of course, much
more expensive.

3ware also makes good RAID controllers, with better Linux
support. Under Windows, that does not matter of course.

Arno
 >> Stay informed about: raid vs network performance 

On Thu, 13 Dec 2007 21:38:20 GMT, 3dO <noaddress.RemoveThis@nomail.com>
wrote:

>ok, this is a long one but i really am clueless on this one
>
>two week ago i set up a "fileserver", as i'm a visual nut and not a
>server nut i went for a simple xp pro machine with a raidcontroller:
>asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
>raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
>spinpoint t166 160GB as systemdisk.
>
>now when i boot up the system the disks make a loud click right after
>spinup, not having any experience with samsungs, i don't know if this is
>normal behaviour for these kind of disks, it sounds scary when you
>always worked with seagates. (they're very silent after this)
>
>the raid 5 setup on the promise card -which is pci with a default and
>unchangeable stripe size of 16KB - is the real problem

So even in the RAID bios, not the software management app in
windows, you can only use 16K?

Is the PCI bus 66 or 33MHz? I suppose this is a desktop
board so it's 33MHz. Yes that would be a bottleneck but not
yet, the PCI bus (assuming you have no other high bandwidth
devices on it) can easily hit 100MB/s and a little more.


>when i work with a 4.01GB video dvd image i get the following transferrates:
>
>localWRITE: copy from systemdisk to raid5 array : 28.9MB/s sustained
>localREAD : copy from raid5 array to systemdisk : 49.4MB/s sustained

The write to the raid array is too slow, I think the problem
is your software RAID card is putting too much overhead on
the CPU. However you don't tell us what you're doing when
are you "wroking with a ... dvd image", if there is other
processing going on with that image or if it is incidental
that it's a DVD image and you just mean you copied a large
file from one place to another.

Since a Core2Duo isn't exactly a slow processor, you might
try changing the driver for the RAID card, or using RAID1
instead of 5, or getting a hardware raid card.


>netWRITE: copy from network to raid5 array : 14.7MB/s
>netREAD : copy from raid5 array to network : 24.5MB/s

These are also dreadful, considering it must be gigabit
ethernet. The raid card is probably part of a bottleneck
per the factors already mentioned, but either the system on
the other end is also a bottleneck or you have a lot of
network processing overhead. Are you using jumbo frames (do
so on if not)? What's the CPU utilization look like during
these copy periods?



>
>local speeds are a bit acceptable but i mainly use it in network over
>gigabit, and those speeds are just not up to my expectations. they are
>not sustained but more like /\_/\_/\_/\ (graph) in bursts, with bursts
>up to 53MB/s, looks like it's getting network traffic and then stopping
>to write it to the array...
>
>now i already swithed my raidcontroller to another pci slot because i
>noticed that it shared an IRQ with the onboard gigabit network
>controller i use to connect it to my network but that didn't help
>i tried all sorts of NTFS cluster setting on the array partition but to
>no avail
>i tried a raid 10 ... no difference (write was slightly better but not
>great)

If the network chip is sitting on the PCI bus that will
limit bandwidth some, but it would not account for such a
low bottleneck of 24MB/s.



>
>so i roamed the net to find ... nothing
>i really don't know what to do, i have the feeling that there is some
>form of issue with sharing pci resources between the pci raid controller
>and the onboard networkcard

Is the onboard network capable of gigabit? If so, try using
it instead. If not, disable it in the bios. Once disabled
there should be no conflicts if there were any.



>so i went back to the shop and they were
>willing to take the promise raid controller back because i was thinking
>about an areca 1210 pci express raid controller and not very pleased
>with the promise card but... i'm not completely sure that it's not
>another problem that's causing these horrible transfer rates.

First I would break up the array (losing data on it) and
benchmark with only one drive on the card as a single drive
span, and/or a single drive strip if it allows that. Also
try with same drive on the motherboard integral controller
with and without the RAID card in the system. Doing these
things will give you a better idea what the peak performance
of the drive is in the best possible scenario, then how the
RAID card limits that, then how that compares to the worst
case results you mentioned above.

Yes, that replacement card has the parameters that should
make it a reasonable replacement. It removes the CPU
processing for the RAID array, removes the as-yet unseen PCI
bottleneck you would see if it were PCI instead of PCI
Express. You might still use jumbo frames for your network
if you aren't yet.

Contrary to another poster's comment, 1GB of memory is not a
factor in this at all. Not even slightly, I've an old video
capture system with 256MB system memory in it, formerly
128MB, that achieves over 30MB/s with a cheap gigabit card.

IF you had a lot of continual access of same files or
multiple concurrent access, then the filecaching of having a
lot of memory would help, but as of yet you have not
described a use where that will be needed.

Also, if your gigabit ethernet card is really poor that can
account for this. I've found the cheap Realtek based cards
do reasonably for the price but the Via chipped cards do
dreadfully in some systems. Intel or 3Com chipped cards
tend to be slightly faster with lower CPU utilization but it
shouldn't matter so much with this fairly modern system, and
yet a different NIC is a small cost increase relative to
what you're already spending for a better RAID card.
 >> Stay informed about: raid vs network performance 

In comp.sys.ibm.pc.hardware.storage kony <spam.TakeThisOut@spam.com> wrote:
> On Thu, 13 Dec 2007 21:38:20 GMT, 3dO <noaddress.TakeThisOut@nomail.com>
> wrote:

>>ok, this is a long one but i really am clueless on this one
>>
>>two week ago i set up a "fileserver", as i'm a visual nut and not a
>>server nut i went for a simple xp pro machine with a raidcontroller:
>>asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
>>raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
>>spinpoint t166 160GB as systemdisk.
>>
>>now when i boot up the system the disks make a loud click right after
>>spinup, not having any experience with samsungs, i don't know if this is
>>normal behaviour for these kind of disks, it sounds scary when you
>>always worked with seagates. (they're very silent after this)
>>
>>the raid 5 setup on the promise card -which is pci with a default and
>>unchangeable stripe size of 16KB - is the real problem

> So even in the RAID bios, not the software management app in
> windows, you can only use 16K?

> Is the PCI bus 66 or 33MHz? I suppose this is a desktop
> board so it's 33MHz. Yes that would be a bottleneck but not
> yet, the PCI bus (assuming you have no other high bandwidth
> devices on it) can easily hit 100MB/s and a little more.

You forget that for copying data or moving it to a network
card, the data has to go over the bus twice....

Arno
 >> Stay informed about: raid vs network performance 

On 14 Dec 2007 11:46:24 GMT, Arno Wagner <me DeleteThis @privacy.net>
wrote:

>In comp.sys.ibm.pc.hardware.storage kony <spam DeleteThis @spam.com> wrote:
>> On Thu, 13 Dec 2007 21:38:20 GMT, 3dO <noaddress DeleteThis @nomail.com>
>> wrote:
>
>>>ok, this is a long one but i really am clueless on this one
>>>
>>>two week ago i set up a "fileserver", as i'm a visual nut and not a
>>>server nut i went for a simple xp pro machine with a raidcontroller:
>>>asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
>>>raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
>>>spinpoint t166 160GB as systemdisk.
>>>
>>>now when i boot up the system the disks make a loud click right after
>>>spinup, not having any experience with samsungs, i don't know if this is
>>>normal behaviour for these kind of disks, it sounds scary when you
>>>always worked with seagates. (they're very silent after this)
>>>
>>>the raid 5 setup on the promise card -which is pci with a default and
>>>unchangeable stripe size of 16KB - is the real problem
>
>> So even in the RAID bios, not the software management app in
>> windows, you can only use 16K?
>
>> Is the PCI bus 66 or 33MHz? I suppose this is a desktop
>> board so it's 33MHz. Yes that would be a bottleneck but not
>> yet, the PCI bus (assuming you have no other high bandwidth
>> devices on it) can easily hit 100MB/s and a little more.
>
>You forget that for copying data or moving it to a network
>card, the data has to go over the bus twice....
>
>Arno

True, the data goes over it twice but that does not account
for the figures the OP lists. The 100MB/s figure was meant
to contrast with the figure given for the local copy
operation, not the network copy operation... which was still
substantially below the PCI bus' potential bottleneck. The
same is true for the network copy operation, the PCI bus is
not the bottleneck "yet", though it may easily be once OP
has the better RAID card that was linked.

As a frame of reference from a far slower (overall) system
than the OP has, I've an old Celeron 500 fileserver with PCI
software raid card and PCI gigabit NIC that achieves about
35MB/s... and this on a mere 66MHz system memory bus with
integrated video usurping part of that memory bandwidth
(Intel i810 chipset without the discrete video memory chips
onboard) and the poor old CPU at 50% utilization from the
network packets and soft-raid driver. The drives aren't
anything special either, probably no faster than what the OP
is using though accessing the oldest drives in it do reduce
the throughput to closer to 20-25MB/s, IIRC.
 >> Stay informed about: raid vs network performance 

In comp.sys.ibm.pc.hardware.storage kony <spam RemoveThis @spam.com> wrote:
> On 14 Dec 2007 11:46:24 GMT, Arno Wagner <me RemoveThis @privacy.net>
> wrote:

>>In comp.sys.ibm.pc.hardware.storage kony <spam RemoveThis @spam.com> wrote:
>>> On Thu, 13 Dec 2007 21:38:20 GMT, 3dO <noaddress RemoveThis @nomail.com>
>>> wrote:
>>
>>>>ok, this is a long one but i really am clueless on this one
>>>>
>>>>two week ago i set up a "fileserver", as i'm a visual nut and not a
>>>>server nut i went for a simple xp pro machine with a raidcontroller:
>>>>asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
>>>>raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
>>>>spinpoint t166 160GB as systemdisk.
>>>>
>>>>now when i boot up the system the disks make a loud click right after
>>>>spinup, not having any experience with samsungs, i don't know if this is
>>>>normal behaviour for these kind of disks, it sounds scary when you
>>>>always worked with seagates. (they're very silent after this)
>>>>
>>>>the raid 5 setup on the promise card -which is pci with a default and
>>>>unchangeable stripe size of 16KB - is the real problem
>>
>>> So even in the RAID bios, not the software management app in
>>> windows, you can only use 16K?
>>
>>> Is the PCI bus 66 or 33MHz? I suppose this is a desktop
>>> board so it's 33MHz. Yes that would be a bottleneck but not
>>> yet, the PCI bus (assuming you have no other high bandwidth
>>> devices on it) can easily hit 100MB/s and a little more.
>>
>>You forget that for copying data or moving it to a network
>>card, the data has to go over the bus twice....
>>
>>Arno

> True, the data goes over it twice but that does not account
> for the figures the OP lists.

Not entirely, no. The network numbers are pretty bad.
The local copy ones may be due to bus limitations.

> The 100MB/s figure was meant
> to contrast with the figure given for the local copy
> operation, not the network copy operation... which was still
> substantially below the PCI bus' potential bottleneck. The
> same is true for the network copy operation, the PCI bus is
> not the bottleneck "yet", though it may easily be once OP
> has the better RAID card that was linked.

It may just be a slow GbE card. The fastest GbE card
for the PCI bus I have ever used leveled out at 60MB/s.
Slower ones (RTL), make maybe 25MB/s. If you need
the full Gigabit, my experience is that you need either
PCI-X or PCI-E (or equivalent on-board cards).

> As a frame of reference from a far slower (overall) system
> than the OP has, I've an old Celeron 500 fileserver with PCI
> software raid card and PCI gigabit NIC that achieves about
> 35MB/s... and this on a mere 66MHz system memory bus with

Yes, but that typically is a 64 bit bus, i.e. has
4 (!) times the PCI bandwidth.

> integrated video usurping part of that memory bandwidth
> (Intel i810 chipset without the discrete video memory chips
> onboard) and the poor old CPU at 50% utilization from the
> network packets and soft-raid driver. The drives aren't
> anything special either, probably no faster than what the OP
> is using though accessing the oldest drives in it do reduce
> the throughput to closer to 20-25MB/s, IIRC.

As I said, it may just be a slow NIC. With a fast NIC,
the OP should get almost the local copy speeds over
the network.

Arno
 >> Stay informed about: raid vs network performance 

On 15 Dec 2007 00:08:07 GMT, Arno Wagner <me.RemoveThis@privacy.net>
wrote:

>In comp.sys.ibm.pc.hardware.storage kony <spam.RemoveThis@spam.com> wrote:
>> On 14 Dec 2007 11:46:24 GMT, Arno Wagner <me.RemoveThis@privacy.net>
>> wrote:
>
>>>In comp.sys.ibm.pc.hardware.storage kony <spam.RemoveThis@spam.com> wrote:
>>>> On Thu, 13 Dec 2007 21:38:20 GMT, 3dO <noaddress.RemoveThis@nomail.com>
>>>> wrote:
>>>
>>>>>ok, this is a long one but i really am clueless on this one
>>>>>
>>>>>two week ago i set up a "fileserver", as i'm a visual nut and not a
>>>>>server nut i went for a simple xp pro machine with a raidcontroller:
>>>>>asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
>>>>>raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
>>>>>spinpoint t166 160GB as systemdisk.
>>>>>
>>>>>now when i boot up the system the disks make a loud click right after
>>>>>spinup, not having any experience with samsungs, i don't know if this is
>>>>>normal behaviour for these kind of disks, it sounds scary when you
>>>>>always worked with seagates. (they're very silent after this)
>>>>>
>>>>>the raid 5 setup on the promise card -which is pci with a default and
>>>>>unchangeable stripe size of 16KB - is the real problem
>>>
>>>> So even in the RAID bios, not the software management app in
>>>> windows, you can only use 16K?
>>>
>>>> Is the PCI bus 66 or 33MHz? I suppose this is a desktop
>>>> board so it's 33MHz. Yes that would be a bottleneck but not
>>>> yet, the PCI bus (assuming you have no other high bandwidth
>>>> devices on it) can easily hit 100MB/s and a little more.
>>>
>>>You forget that for copying data or moving it to a network
>>>card, the data has to go over the bus twice....
>>>
>>>Arno
>
>> True, the data goes over it twice but that does not account
>> for the figures the OP lists.
>
>Not entirely, no. The network numbers are pretty bad.
>The local copy ones may be due to bus limitations.

This is not likely, these read and write tests were to/fro
the PCI raid card but the other drive was on a southbridge
integral controller that is not logically on the PCI bus.
In other words as I'd written already, the PCI bus will not
limit it untill around 100MB/s or higher, unless there was
other concurrent PCI bus activity but it seems pretty
unlikely as this was a focused test.



>
>> The 100MB/s figure was meant
>> to contrast with the figure given for the local copy
>> operation, not the network copy operation... which was still
>> substantially below the PCI bus' potential bottleneck. The
>> same is true for the network copy operation, the PCI bus is
>> not the bottleneck "yet", though it may easily be once OP
>> has the better RAID card that was linked.
>
>It may just be a slow GbE card. The fastest GbE card
>for the PCI bus I have ever used leveled out at 60MB/s.
>Slower ones (RTL), make maybe 25MB/s.

RT8169 can fairly easily do 40MB/s.


>If you need
>the full Gigabit, my experience is that you need either
>PCI-X or PCI-E (or equivalent on-board cards).

Sure, but it would be a bit of an extreme expense, when
talking about a RAID5 of consumer grade 7K2 RPM SATA drives.



>
>> As a frame of reference from a far slower (overall) system
>> than the OP has, I've an old Celeron 500 fileserver with PCI
>> software raid card and PCI gigabit NIC that achieves about
>> 35MB/s... and this on a mere 66MHz system memory bus with
>
>Yes, but that typically is a 64 bit bus, i.e. has
>4 (!) times the PCI bandwidth.

In practice, benchmarks of that era showed these platforms
achieving about 200MB/s (actually a little less), minus the
bandwidth used by the integrated video and other system
operations.


>
>> integrated video usurping part of that memory bandwidth
>> (Intel i810 chipset without the discrete video memory chips
>> onboard) and the poor old CPU at 50% utilization from the
>> network packets and soft-raid driver. The drives aren't
>> anything special either, probably no faster than what the OP
>> is using though accessing the oldest drives in it do reduce
>> the throughput to closer to 20-25MB/s, IIRC.
>
>As I said, it may just be a slow NIC. With a fast NIC,
>the OP should get almost the local copy speeds over
>the network.
>
>Arno

A different NIC may have some benefit but that much is not
a certainty. At some point the PCI bus will be a bottleneck
as these drives may be able to sustain over 50MB/s in a
local copy.
http://www.anandtech.com/printarticle.aspx?i=3031
 >> Stay informed about: raid vs network performance 

kony wrote:
> On Thu, 13 Dec 2007 21:38:20 GMT, 3dO <noaddress.DeleteThis@nomail.com>
> wrote:
>
>> ok, this is a long one but i really am clueless on this one
>>
>> two week ago i set up a "fileserver", as i'm a visual nut and not a
>> server nut i went for a simple xp pro machine with a raidcontroller:
>> asus p5k-vm with intel c2d e4300 - 1GB ram, promise fasttrak tx4310 PCI
>> raid controller, 4 samsung spinpoint t166 500GB (raid5) and 1 samsung
>> spinpoint t166 160GB as systemdisk.
>>
>> now when i boot up the system the disks make a loud click right after
>> spinup, not having any experience with samsungs, i don't know if this is
>> normal behaviour for these kind of disks, it sounds scary when you
>> always worked with seagates. (they're very silent after this)
>>
>> the raid 5 setup on the promise card -which is pci with a default and
>> unchangeable stripe size of 16KB - is the real problem
>
> So even in the RAID bios, not the software management app in
> windows, you can only use 16K?
>
> Is the PCI bus 66 or 33MHz? I suppose this is a desktop
> board so it's 33MHz. Yes that would be a bottleneck but not
> yet, the PCI bus (assuming you have no other high bandwidth
> devices on it) can easily hit 100MB/s and a little more.
>
>
>> when i work with a 4.01GB video dvd image i get the following transferrates:
>>
>> localWRITE: copy from systemdisk to raid5 array : 28.9MB/s sustained
>> localREAD : copy from raid5 array to systemdisk : 49.4MB/s sustained
>
> The write to the raid array is too slow, I think the problem
> is your software RAID card is putting too much overhead on
> the CPU. However you don't tell us what you're doing when
> are you "wroking with a ... dvd image", if there is other
> processing going on with that image or if it is incidental
> that it's a DVD image and you just mean you copied a large
> file from one place to another.
>
> Since a Core2Duo isn't exactly a slow processor, you might
> try changing the driver for the RAID card, or using RAID1
> instead of 5, or getting a hardware raid card.
>
>
>> netWRITE: copy from network to raid5 array : 14.7MB/s
>> netREAD : copy from raid5 array to network : 24.5MB/s
>
> These are also dreadful, considering it must be gigabit
> ethernet. The raid card is probably part of a bottleneck
> per the factors already mentioned, but either the system on
> the other end is also a bottleneck or you have a lot of
> network processing overhead. Are you using jumbo frames (do
> so on if not)? What's the CPU utilization look like during
> these copy periods?
>
>
>
>> local speeds are a bit acceptable but i mainly use it in network over
>> gigabit, and those speeds are just not up to my expectations. they are
>> not sustained but more like /\_/\_/\_/\ (graph) in bursts, with bursts
>> up to 53MB/s, looks like it's getting network traffic and then stopping
>> to write it to the array...
>>
>> now i already swithed my raidcontroller to another pci slot because i
>> noticed that it shared an IRQ with the onboard gigabit network
>> controller i use to connect it to my network but that didn't help
>> i tried all sorts of NTFS cluster setting on the array partition but to
>> no avail
>> i tried a raid 10 ... no difference (write was slightly better but not
>> great)
>
> If the network chip is sitting on the PCI bus that will
> limit bandwidth some, but it would not account for such a
> low bottleneck of 24MB/s.
>
>
>
>> so i roamed the net to find ... nothing
>> i really don't know what to do, i have the feeling that there is some
>> form of issue with sharing pci resources between the pci raid controller
>> and the onboard networkcard
>
> Is the onboard network capable of gigabit? If so, try using
> it instead. If not, disable it in the bios. Once disabled
> there should be no conflicts if there were any.
>
>
>
>> so i went back to the shop and they were
>> willing to take the promise raid controller back because i was thinking
>> about an areca 1210 pci express raid controller and not very pleased
>> with the promise card but... i'm not completely sure that it's not
>> another problem that's causing these horrible transfer rates.
>
> First I would break up the array (losing data on it) and
> benchmark with only one drive on the card as a single drive
> span, and/or a single drive strip if it allows that. Also
> try with same drive on the motherboard integral controller
> with and without the RAID card in the system. Doing these
> things will give you a better idea what the peak performance
> of the drive is in the best possible scenario, then how the
> RAID card limits that, then how that compares to the worst
> case results you mentioned above.
>
> Yes, that replacement card has the parameters that should
> make it a reasonable replacement. It removes the CPU
> processing for the RAID array, removes the as-yet unseen PCI
> bottleneck you would see if it were PCI instead of PCI
> Express. You might still use jumbo frames for your network
> if you aren't yet.
>
> Contrary to another poster's comment, 1GB of memory is not a
> factor in this at all. Not even slightly, I've an old video
> capture system with 256MB system memory in it, formerly
> 128MB, that achieves over 30MB/s with a cheap gigabit card.
>
> IF you had a lot of continual access of same files or
> multiple concurrent access, then the filecaching of having a
> lot of memory would help, but as of yet you have not
> described a use where that will be needed.
>
> Also, if your gigabit ethernet card is really poor that can
> account for this. I've found the cheap Realtek based cards
> do reasonably for the price but the Via chipped cards do
> dreadfully in some systems. Intel or 3Com chipped cards
> tend to be slightly faster with lower CPU utilization but it
> shouldn't matter so much with this fairly modern system, and
> yet a different NIC is a small cost increase relative to
> what you're already spending for a better RAID card.

Ok thanks a lot for all the info of all of you
To comment on your questions Kony:

> So even in the RAID bios, not the software management app in
> windows, you can only use 16K?
Yes, the 16K is a "limit" of the card, you cannot change it, manual and
promise website confirm this

> The write to the raid array is too slow, I think the problem
> is your software RAID card is putting too much overhead on
> the CPU. However you don't tell us what you're doing when
> are you "wroking with a ... dvd image", if there is other
> processing going on with that image or if it is incidental
> that it's a DVD image and you just mean you copied a large
> file from one place to another.
It is actually just copying an already rendered DVD image, so just one
file from a to b over network, no background processes or anything.

> These are also dreadful, considering it must be gigabit
> ethernet. The raid card is probably part of a bottleneck
> per the factors already mentioned, but either the system on
> the other end is also a bottleneck or you have a lot of
> network processing overhead. Are you using jumbo frames (do
> so on if not)? What's the CPU utilization look like during
> these copy periods?
The system i did my tests from is an intel c2d e6600, 320 seagate sata
disk, 2gigs of ram, gfx 8800gtx and windows vista premium (work horse
which doubles as a game animal) so i doubt that this would be an issue.
CPU loads on the raid5 machine during copy (local or network) never
exceeds 12% on either core. (note: with avg antivirus running, turning
this off does not have any impact on performance)

> Is the onboard network capable of gigabit? If so, try using
> it instead. If not, disable it in the bios. Once disabled
> there should be no conflicts if there were any.
Maybe an unclarity from my part, i am using the onboard gigabit network


So now i'm really thinking about trading in the -indeed- softraid card
for an areca, what would mean spending some 150 euros extra while
looking for prices for an intel Gbit networdcard over pci express, these
big boys are going over the counter here where i live for some 90 euros
which compared to the 150 of the areca is quite hefty...

Maybe another idea from you guys ? but i'm really thinking, after
reading all posts here, that the promise raid card is just a piece of
BEEP causing the problems.
 >> Stay informed about: raid vs network performance 

Ok thanks a lot for all the info of all of you
To comment on your questions Kony:

> So even in the RAID bios, not the software management app in
> windows, you can only use 16K?
Yes, the 16K is a "limit" of the card, you cannot change it, manual and
promise website confirm this

> The write to the raid array is too slow, I think the problem
> is your software RAID card is putting too much overhead on
> the CPU. However you don't tell us what you're doing when
> are you "wroking with a ... dvd image", if there is other
> processing going on with that image or if it is incidental
> that it's a DVD image and you just mean you copied a large
> file from one place to another.
It is actually just copying an already rendered DVD image, so just one
file from a to b over network, no background processes or anything.

> These are also dreadful, considering it must be gigabit
> ethernet. The raid card is probably part of a bottleneck
> per the factors already mentioned, but either the system on
> the other end is also a bottleneck or you have a lot of
> network processing overhead. Are you using jumbo frames (do
> so on if not)? What's the CPU utilization look like during
> these copy periods?
The system i did my tests from is an intel c2d e6600, 320 seagate sata
disk, 2gigs of ram, gfx 8800gtx and windows vista premium (work horse
which doubles as a game animal) so i doubt that this would be an issue.
CPU loads on the raid5 machine during copy (local or network) never
exceeds 12% on either core. (note: with avg antivirus running, turning
this off does not have any impact on performance)

> Is the onboard network capable of gigabit? If so, try using
> it instead. If not, disable it in the bios. Once disabled
> there should be no conflicts if there were any.
Maybe an unclarity from my part, i am using the onboard gigabit network


So now i'm really thinking about trading in the -indeed- softraid card
for an areca, what would mean spending some 150 euros extra while
looking for prices for an intel Gbit networdcard over pci express, these
big boys are going over the counter here where i live for some 90 euros
which compared to the 150 of the areca is quite hefty...

Maybe another idea from you guys ? but i'm really thinking, after
reading all posts here, that the promise raid card is just a piece of
BEEP causing the problems.
 >> Stay informed about: raid vs network performance 

On Sat, 15 Dec 2007 22:00:06 GMT, 3dO <noaddress.TakeThisOut@nomail.com>
wrote:


> > These are also dreadful, considering it must be gigabit
> > ethernet. The raid card is probably part of a bottleneck
> > per the factors already mentioned, but either the system on
> > the other end is also a bottleneck or you have a lot of
> > network processing overhead. Are you using jumbo frames (do
> > so on if not)? What's the CPU utilization look like during
> > these copy periods?

>The system i did my tests from is an intel c2d e6600, 320 seagate sata
>disk, 2gigs of ram, gfx 8800gtx and windows vista premium (work horse
>which doubles as a game animal) so i doubt that this would be an issue.
>CPU loads on the raid5 machine during copy (local or network) never
>exceeds 12% on either core. (note: with avg antivirus running, turning
>this off does not have any impact on performance)

You are listing things that have nothing to do with
performance in this test. 2GB of ram, video, don't matter.
Vista makes it slower too but it shouldn't make it this
slow. Even so, I would retest under WinXP because actually
Vista is known to have some severe hesitations on some
systems but it is generally when dealing with a lot of
files, not just copying one large file.

You did not mention if you are using jumbo frames. I
suppose it should also be said, that of course with jumbo
frames your switches (if there is one) have to also support
it, and further there are some TCP/IP tweaks (Google will
find them) that benefit LAN performance over gigabit
ethernet.


>
> > Is the onboard network capable of gigabit? If so, try using
> > it instead. If not, disable it in the bios. Once disabled
> > there should be no conflicts if there were any.

>Maybe an unclarity from my part, i am using the onboard gigabit network
>
>

Can you determine if it is logically sitting on the PCI bus?
This might be useful to know as it effects available PCI bus
bandwidth during network copying.



>So now i'm really thinking about trading in the -indeed- softraid card
>for an areca, what would mean spending some 150 euros extra while
>looking for prices for an intel Gbit networdcard over pci express, these
>big boys are going over the counter here where i live for some 90 euros
>which compared to the 150 of the areca is quite hefty...
>
>Maybe another idea from you guys ? but i'm really thinking, after
>reading all posts here, that the promise raid card is just a piece of
>BEEP causing the problems.


As already suggested, you should benchmark in other
scenarios. Connect a single drive span to the promise card
and test local and network copy. Test the motherboard
controller connected drive doing network copy. Compare the
results to see how much better each scenario is.

Yes, ultimately the fastest performing solution will be to
replace the PCI software raid card with a hardware PCI
Express version, and the same for the gigabit ethernet card.
The reason question is how much benefit at what cost, and
what your uses are since some don't need utmost performance.
I mention this because many systems can use cheaper hardware
and still achieve 25-30MB/s regularly, though RAID5 on a
software card is probably not a good idea.
 >> Stay informed about: raid vs network performance 

On Sun, 16 Dec 2007 04:41:44 -0500, kony <spam.DeleteThis@spam.com>
wrote:


>Not true, the vast majority did reach peak throughtput, with
>exception of some Via and early skt 370 Sis chipsets. The
>rest did quite well at such a meager task. I have build
>quite a few filesevers from old legacy systems, I would even
>consider them ideal for this use if someone didn't need over
>5MB/s,

I meant "didn't need over 35MB's"

The one noteworthy thing about the now aged and seemingly
slow PCI bus was, if the chipset or board designers didn't
make terrible mistake, it really was capable of 100MB/s if
not more... even on an otherwise sluggish system.

Once upon a time ago, even with a meager PII/233 with ATA33
interfaced HDD, I was seeing a little over 25MB/s using a
RT8169 gigabit ethernet card. While that is not impressive,
it's still higher than the 24MB/s that was reported.
 >> Stay informed about: raid vs network performance 

On 16 Dec 2007 16:04:55 GMT, Arno Wagner <me.TakeThisOut@privacy.net>
wrote:


>>>>>Slower ones (RTL), make maybe 25MB/s.
>>>
>>>> RT8169 can fairly easily do 40MB/s.
>>>
>>>Depends very much on the board and the other components
>>>in use. I have seen speeds as low as 240Mbit/s and that
>>>was with UDP streaming, not TCP.
>
>
>> ... Then it's not RT8169 that is the bottleneck and the
>> comment about 25MB/s is just invalid.
>
>
>240Mbit/s = 25 MB/s

And? It doesn't change that I have systems with RT8169
which achieve more than 25MB/s.



>
>> I have, maybe half a dozen systems using these RT8169
>> cards, they are not at all limited to the low throughput the
>> OP is seeing, BUT, on any I bothered to benchmark, I did
>> ensure better jumbo frame TCP/IP settings.
>
>There are different chip generations and the combination
>of mainboard and chip also matters.

Some, certainly with a few Via chipsets which had poor PCI
bandwidth that was choking data flow to the NIC. As for
pure chipset - NIC interactions, it is very rare and even
more often attributable to an OS problem rather than
mainboard problem.



>
>> Also, at some point realtek limited jumbo frame support to
>> around 7K instead of 9K, so that limit is worth
>> consideration but in real world use, 7K is not going to
>> bottleneck anywhere near this value reported... maybe if it
>> were a 80486 system or Pentium 1.
>
>If you can use Jumbo-frames. Depends.

Most can at least use 4K frames if not 7-9K. The majority
of networking equipment will pass more than 1.5K.


>>>Yes, and PCI busses of that aera can often not reach more
>>>than 70-80MB/s.
>
>
>> Not true, the vast majority did reach peak throughtput, with
>> exception of some Via and early skt 370 Sis chipsets.
>
>So? Then I must have consitently bought the wrong hardware....

Many of us did buy a chipset or two that had this problem,
but by midway into the Pentium 3 era, it was mostly reserved
to Via chipsets as Sis had then covercome it. Intel had
great PCI throughput much further back, I can't put my
finger on exactly when it was more than 80MB/s but certainly
by the 440LX/P2 era, and probably before that.



>> I can say that any and all systems I have used Realtek 8169
>> based PCI cards on, did not have any substantil bottleneck
>> based on that card. What I really mean is, this is a cheap
>> card and used when other things are budget constained too,
>> that it can easily, always reach over 30Mb/s when there is
>
>I assume you mean 30MB/s?

Yes, I mistyped 30MB/s.


>> IOW, if the drive can do in excess of 50MB/s, which
>> benchmarks show it can, then it's 50MB/s over the PCI bus to
>> memory, then back again to the PCI NIC. While the PCI bus
>> can achieve 100Mb/s, it can't do much more without at least
>
>Again, woujd that be 100MB/s?

Yep, but wouldn't that have been obvious by my prior
statements?
 >> Stay informed about: raid vs network performance