Wednesday, January 18, 2012

Divide single Dell NIC into multiple NICs, going from 2 to 8 nics per blade Dell Blade Server

I was looking for the Dell equivolent to HP Flex-10’s FlexFabric Adapter.  In dell speak this is called “independent NIC partitioning“ or just “NIC partitioning” or NPAR. 

First let me give you some background into my new system.  It consists of M1000E blade cabinets, M710HD Blade Servers with Broadcom 57712-k 10GbE 2P nics, and PowerConnect M8024-k Cabinet Switches.  My goal is to turn two 10GB NIC’s into four NIC’s made up of two 1GB NIC’s and two 9GB NIC’s

Before you connect to iDRAC, If you want to use your mouse, you must set the Mouse Mode to USC\Diags (also don’t do this through remote desktop)  Make this change from the iDRAC GUI.  I always change the media to attach or I won’t be able to install ESX later.

image

Apply the setting, make sure to wait for the confirmation, otherwise it didn’t happen.

To make the actual NIC Partitioning changes, you must use the ""Dell Unified Server Configurator”, which being the noob I am, I tried to find for download, but apparently you access it during the blade server boot by pressing F10 to access the UEFI (System Services")

image

Then you will see something like this when it boots to UEFI

image

Select “Hardware Configuration”, you can do that with your mouse or the arrows on the keyboard. 

image

Then choose tab over to HII Advanced Configuration

image

You will need to do the following twice, once per nic.

image

Then Device Configuration Menu

image

Change Disabled to “Enabled”

image

After you hit back, you will see a new options, “NIC Partitioning Configuration Menu”, select it now.

image

Then Select “Global Bandwidth Allocation Menu”

image

Tab doesn’t work on the next page, some person decided to make tab only go back and forth between the top row and the back button.  However, luckily you can use your UP and Down arrows to move between rows.

Unfortunately it doesn’t appear you can Partition a NIC into less than 4, I only want 2, so using that methodology I will create one with 90 and one with 10, the other two I will give 1 to because I can’t give them 0.  You are allowed to over allocate the Maximum Bandwidth.  1%=100MB, so the find 2 NIC’s I will create will be 100MB adapters I won’t end up using.

Type in your number, then hit enter, then choose the next row.

HINT: if using the mouse and it gets a bit squirrly, Play with “ALT-C” and toggle Hide Local Cursor on and off to improve mouse response.

image

Choose Back, then Back, then Finish,  It will prompt you to Save, of course say yes, then repeat for your next adapter.

image

After you have done both NIC’s, Choose back, then Exit and Reboot.

Here comes the really fun part.  After installing and booting into ESX, it appears that only the 2nd NIC took the partitioning command as the first one is still at 25/25/25/25 and the second one is 10/90/1/1 correctly.  I made the changes again for the first NIC and rebooted again, and it works.  I verified this over 3 different new blades, if anyone knows the solution, I’d love to hear it.

 

Credits, I found the information for this article HERE

http://www.dell.com/downloads/global/products/pwcnt/en/broadcom-npar-users-manual.pdf

http://www.dell.com/downloads/global/products/pedge/en/Dell-Broadcom-NPAR-White-Paper.pdf

http://www.dell.com/us/enterprise/p/broadcom-netxtremeii-57712-k/pd

6 comments:

Roity said...

http://www.qlogic.com/OEMPartnerships/Dell/Documents/Converged/Dell_QLogic_NPAR.pptx

That's for the QLogic, and you can do all the config for it within the QLogic boot configuration (no need to go into DELL USC), refer to slide 9 & 10.

Flo, Mélinda & Noémie said...

Hello, Thanks for this is a very good explanation..
I've just one question on NPAR, what about Vlan ? The switch only see 1 port for 4 virtual ports. How can you affect Vlan like iSCSI to only one virtual port ?
Do you need to use M8424-k to do this, as this swith should be aware of de several virtual ports ?
Regards,

Brian Smith said...

Flo,
I wrote a separate article about this http://bsmith9999.blogspot.com/2012/01/setup-vlan-tagging-for-esxi-on-dell.html . Good Luck!

Matthew Newton said...

Thanks for the info--it's hard to cut through the market-speak to get to what the product actually does. 5 minutes on this article is equivalent to an hour on Dell's site!

Greg Jacknow said...

Just curious...Why would you ever want/need to do this if your servers are VMware servers?

Could you not just create a vSwitch pointing at the 10Gb adapters and using multiple port groups to access the different VLANs?

Brian Smith said...

There are several reasons. You may not be able to afford licensing that includes vDS's (to get the NIOC features). Also, some people just like to control things at various levels i.e. Hardware vs Software.