(Virtual Functions) |
|||
| (17 intermediate revisions by 6 users not shown) | |||
| Line 1: | Line 1: | ||
I see that the on-board NICs are enumerated 1, 2, 3, & 4. What does this mean for motherboards that have more than 4 NICs? We use ATCA boards that consistently have 6 to 8 on-board NICs. How would these be enumerated under the new system? | I see that the on-board NICs are enumerated 1, 2, 3, & 4. What does this mean for motherboards that have more than 4 NICs? We use ATCA boards that consistently have 6 to 8 on-board NICs. How would these be enumerated under the new system? | ||
2011-01-31 mdomsch: There is no limit to the number of on-board NICs. I used the example of 4, but if you have more, say N, and SMBIOS or PIRQ reports those all as on-board, they will be named em1..emN. | |||
== Avoiding this feature == | == Avoiding this feature == | ||
Will there be any way to disable this feature on fresh install? (e.g. removing udev-script or disabling it in script in /etc/sysconf). I want to keep with Linux interface naming standards based on protocol (eth, ppp, tun, tap) and not with half-half mess made by this feature. Thank you for reply. | Will there be any way to disable this feature on fresh install? (e.g. removing udev-script or disabling it in script in /etc/sysconf). I want to keep with Linux interface naming standards based on protocol (eth, ppp, tun, tap) and not with half-half mess made by this feature. Thank you for reply. | ||
2011-01-28 Yes, we are adding a kernel command line option to disable biosdevname from running at install time; you can also uninstall the biosdevname package after install. -mdomsch | |||
2012-08-25 I've been burned too. This is particularly painful for scripts that execute on multiple distributions - especially fedora AND embedded. Isn't there a better way? Can one create an alias '''name''' for a network device? E.g. with a real device, one simply symlinks /dev/standardname -> /dev/wierdname. | |||
If we could do this with network devices, this aggravation would disappear. This would mean that eth0 -> p2p1 (or whatever), and would imply that eth0:n -> p2p1:n. Scripts could continue to use/search for eth device names, which would be stable because the consistent name is stable. No udev rules or boot parameter magic; everyone gets what (s)he wants. | |||
I would argue that for compatibility, the eth alias should be created by default. In the common case - one network adapter - it will always be correct. With multiple adapters, when no alias is present, assign eth aliases in detection order - if this is wrong, it can be adjusted manually. | |||
Right now, I'd settle for a manual command equivalent to ''ln -s'' in ''/dev''... Then I could report the failure to do this by default as a bug :-) | |||
-tlhackque | |||
== Avoiding it at all! == | == Avoiding it at all! == | ||
This new feature can introduce much more problems then it solves. Looks scary for me as sysadmin and programmer. Well, a lot of net scripts, iptables rules and other stuff involved. Yes, I use variables in bash, and other tricks that will help me to survive such global change. But the question is simple: Is a such little benefit worth of such huge bunch of stupid work for all Fedora users? | This new feature can introduce much more problems then it solves. Looks scary for me as sysadmin and programmer. Well, a lot of net scripts, iptables rules and other stuff involved. Yes, I use variables in bash, and other tricks that will help me to survive such global change. But the question is simple: Is a such little benefit worth of such huge bunch of stupid work for all Fedora users? | ||
2012-03-14: I would argue that this feature causes problems for people while providing little to no benefit. It should not have been enabled by default. I have been using Fedora since the very first release so I speak from experience here. Not only does this change network device naming that everyone is already familiar with, it does not always work as expected. I have a motherboard (SuperMicro C2SBC-Q) with two embedded NICs and one shows up as em1 and one shows up as p5p1 even though both are embedded in the motherboard. How does this new naming convention help me figure out which one is which? It doesn't. I still have to use ethtool to figure out which one is which. Also, as noted in the VM section, the special VM exception doesn't seem to apply to VirtualBox. You still end up with p<#>p<#> naming. This feature should have been optional, not a default. If Dell wants to do things a certain way, let them, but don't affect the whole community for something that only works for specific hardware. -- jake1138 | |||
== Optimizing for an edge case? == | == Optimizing for an edge case? == | ||
| Line 14: | Line 31: | ||
Frankly using mii-tool seems much more reliable and easier to identify the Nics. and then just label them in the ifcfg-* files. And what's to say that Motherboards are going to consistently have their ports named and numbered in a consistent order either? the udev changes with the 70-persistent-net* file seems to have made nic handling on linux SOO much better when adding multiple nics in a system. Especially since once you find out which nic is which you can just edit the file and name them where you want and reboot. | Frankly using mii-tool seems much more reliable and easier to identify the Nics. and then just label them in the ifcfg-* files. And what's to say that Motherboards are going to consistently have their ports named and numbered in a consistent order either? the udev changes with the 70-persistent-net* file seems to have made nic handling on linux SOO much better when adding multiple nics in a system. Especially since once you find out which nic is which you can just edit the file and name them where you want and reboot. | ||
--[[User:Urkle|Urkle]] 21:34, 26 January 2011 (UTC) | --[[User:Urkle|Urkle]] 21:34, 26 January 2011 (UTC) | ||
2011-01-31: Yes, you can always do this, both with and without biosdevname. Good luck doing it for more than a few servers. | |||
== Virtual Function Devices? == | == Virtual Function Devices? == | ||
Newer server NICs often have a feature called Virtual Functions. This is a virtual NIC bridged to a real NIC port. The virtual function can then be assigned to a virtual machine using Vt-d. This avoids the need for context switches during send/receive and increases network speed for virtual machines. | Newer server NICs often have a feature called Virtual Functions. This is a virtual NIC bridged to a real NIC port. The virtual function can then be assigned to a virtual machine using Vt-d. This avoids the need for context switches during send/receive and increases network speed for virtual machines. | ||
How will the virtual functions be called? | How will the virtual functions be called (on the host, while the vm is not active)? | ||
Will they get a constant name, even if their MACs are random? | Will they get a constant name, even if their MACs are random? | ||
| Line 24: | Line 43: | ||
I ask this because the naming of vfs was a big mess when vfs were first introduced. | I ask this because the naming of vfs was a big mess when vfs were first introduced. | ||
--[[User:Gvegidy|Gvegidy]] 21:41, 26 January 2011 (UTC) | --[[User:Gvegidy|Gvegidy]] 21:41, 26 January 2011 (UTC) | ||
2011-01-28: Yes, virtual functions (VFs) on SR-IOV devices will have a suffix of "_<number>" appended to their name. For example, pci2#1_63 is the card in PCI slot 2, port 1, VF 63. These are exposed in the Host, and may then be direct-assigned to a Guest (using VT-d). The MACs are random, but the naming convention will remain. | |||
--[[User:Mdomsch|Mdomsch]] 28 January 2011 | |||
== Benefit for ordinary user == | |||
I see benefits for this only in special configurations, like servers with big number of network cards, or stateless, or PXE boot, or cluster, etc. For ordinary users it brings only subtle benefits and much more inconvenience. May be this feature shouldn't be enabled by default? | |||
2011-01-31 mdomsch: "ordinary users" (e.g. Desktop users?) are presumably using NetworkManager, which doesn't care what the device name actually is, it finds devices and uses them. I also disagree about "much more inconvenience". Please elaborate. | |||
Well, ok, this feature is not concern those who are using only NetworkManager and nothing more. But, quite big portion of people in Fedora are experimenting or just going to get some experience. This feature, IMHO, will make learning threshold higher for them as it will invalidate many of manuals found in the Internet for networking things. For example, creating virtual machines in libvirt with bridging (NM still doesn't support this, AFAIK), firewalling/traffic control/routing, or just binding network daemon to specific interface, etc. | |||
For admins and people who help on chats or forums this is also true, but they have enough experience to grasp what is going on. --[[User:Atorkhov|Atorkhov]] 15:51, 31 January 2011 (UTC) | |||
== Why make an exception for VMs/Guests? == | |||
The docs say you will continue to use the ethX naming convention when running as a VM guest. Why? The OS should should exhibit consistant behavior, whether virtualized or not. VMs have virtual hardware that needs to be matched up to detected OS devices in just the same way physical systems do. And how would you reliably "know" that the OS is virtualized? | |||
It's common to write scripts that work across both virtualized and non-virtualized systems. If you continue to use the ethX on virtual, but the new convention on physical, automating NIC changes will require adding a virtualization check and split paths for either case. It also complicates P2V server migrations. | |||
This seems like an ill-conceived change, but if you're going to do it, please consider keeping parity between virtualized and non-virtualized systems. | |||
2011-02-23: one can detect running in a VM with a simple cpuid instruction. However, each VM platform (KVM, Xen, ESX, Hyper-V, ...) exposes its NICs in completely different manners (some are in slot 2, some in slot 3, some embedded, some with no information at all; and you can have all of these in a single VM as well). At least with this change, biosdevname doesn't behave differently for each platform; it just does nothing. | |||
P2V migrations change their MAC addresses during conversion already, so you must be able to account for such changes with the current udev name<->MAC mapping. You can continue using name<->MAC mapping to get names you want, in which case biosdevname won't be invoked. | |||
2011-04-29: If "each VM platform (KVM, Xen, ESX, Hyper-V, ...) exposes its NICs in completely different manners" is the reasoning, how is this any different than saying that every motherboard exposes its NICs in completely different manners? This sounds like a great argument against "consistent network device naming" really being consistent. | |||
2011-04-29: VM BIOSes have so far not exposed the information via either the ACPI _DSM recently added to the spec, or to SMBIOS 2.6 fields. They were only exposing it (if at all) via the PCI IRQ Routing Table, and then, in a highly inconsistent manner. Furthermore, biosdevname is about physicality of a device location, which has no meaning in a VM. It's not about logical "what is this NIC connected to", which is really what you want for a VM. I think it would be cool to have a way for a VM to know which vSwitch or bridge a vNIC is connected to, and to name accordingly, but that is outside the scope of biosdevname. Solutions to that problem are welcome. | |||
2012-02-06: There does not appear to be an exception in the case of VirtualBox. I just installed Fedora 16 and it shows the interface as p2p1. | |||
Latest revision as of 19:10, 25 August 2012
I see that the on-board NICs are enumerated 1, 2, 3, & 4. What does this mean for motherboards that have more than 4 NICs? We use ATCA boards that consistently have 6 to 8 on-board NICs. How would these be enumerated under the new system?
2011-01-31 mdomsch: There is no limit to the number of on-board NICs. I used the example of 4, but if you have more, say N, and SMBIOS or PIRQ reports those all as on-board, they will be named em1..emN.
Avoiding this feature
Will there be any way to disable this feature on fresh install? (e.g. removing udev-script or disabling it in script in /etc/sysconf). I want to keep with Linux interface naming standards based on protocol (eth, ppp, tun, tap) and not with half-half mess made by this feature. Thank you for reply.
2011-01-28 Yes, we are adding a kernel command line option to disable biosdevname from running at install time; you can also uninstall the biosdevname package after install. -mdomsch
2012-08-25 I've been burned too. This is particularly painful for scripts that execute on multiple distributions - especially fedora AND embedded. Isn't there a better way? Can one create an alias name for a network device? E.g. with a real device, one simply symlinks /dev/standardname -> /dev/wierdname.
If we could do this with network devices, this aggravation would disappear. This would mean that eth0 -> p2p1 (or whatever), and would imply that eth0:n -> p2p1:n. Scripts could continue to use/search for eth device names, which would be stable because the consistent name is stable. No udev rules or boot parameter magic; everyone gets what (s)he wants.
I would argue that for compatibility, the eth alias should be created by default. In the common case - one network adapter - it will always be correct. With multiple adapters, when no alias is present, assign eth aliases in detection order - if this is wrong, it can be adjusted manually.
Right now, I'd settle for a manual command equivalent to ln -s in /dev... Then I could report the failure to do this by default as a bug :-)
-tlhackque
Avoiding it at all!
This new feature can introduce much more problems then it solves. Looks scary for me as sysadmin and programmer. Well, a lot of net scripts, iptables rules and other stuff involved. Yes, I use variables in bash, and other tricks that will help me to survive such global change. But the question is simple: Is a such little benefit worth of such huge bunch of stupid work for all Fedora users?
2012-03-14: I would argue that this feature causes problems for people while providing little to no benefit. It should not have been enabled by default. I have been using Fedora since the very first release so I speak from experience here. Not only does this change network device naming that everyone is already familiar with, it does not always work as expected. I have a motherboard (SuperMicro C2SBC-Q) with two embedded NICs and one shows up as em1 and one shows up as p5p1 even though both are embedded in the motherboard. How does this new naming convention help me figure out which one is which? It doesn't. I still have to use ethtool to figure out which one is which. Also, as noted in the VM section, the special VM exception doesn't seem to apply to VirtualBox. You still end up with p<#>p<#> naming. This feature should have been optional, not a default. If Dell wants to do things a certain way, let them, but don't affect the whole community for something that only works for specific hardware. -- jake1138
Optimizing for an edge case?
It seems to me this is simply optimizing for an edge case instead of the general use.
Frankly using mii-tool seems much more reliable and easier to identify the Nics. and then just label them in the ifcfg-* files. And what's to say that Motherboards are going to consistently have their ports named and numbered in a consistent order either? the udev changes with the 70-persistent-net* file seems to have made nic handling on linux SOO much better when adding multiple nics in a system. Especially since once you find out which nic is which you can just edit the file and name them where you want and reboot. --Urkle 21:34, 26 January 2011 (UTC)
2011-01-31: Yes, you can always do this, both with and without biosdevname. Good luck doing it for more than a few servers.
Virtual Function Devices?
Newer server NICs often have a feature called Virtual Functions. This is a virtual NIC bridged to a real NIC port. The virtual function can then be assigned to a virtual machine using Vt-d. This avoids the need for context switches during send/receive and increases network speed for virtual machines.
How will the virtual functions be called (on the host, while the vm is not active)?
Will they get a constant name, even if their MACs are random?
I ask this because the naming of vfs was a big mess when vfs were first introduced. --Gvegidy 21:41, 26 January 2011 (UTC)
2011-01-28: Yes, virtual functions (VFs) on SR-IOV devices will have a suffix of "_<number>" appended to their name. For example, pci2#1_63 is the card in PCI slot 2, port 1, VF 63. These are exposed in the Host, and may then be direct-assigned to a Guest (using VT-d). The MACs are random, but the naming convention will remain. --Mdomsch 28 January 2011
Benefit for ordinary user
I see benefits for this only in special configurations, like servers with big number of network cards, or stateless, or PXE boot, or cluster, etc. For ordinary users it brings only subtle benefits and much more inconvenience. May be this feature shouldn't be enabled by default?
2011-01-31 mdomsch: "ordinary users" (e.g. Desktop users?) are presumably using NetworkManager, which doesn't care what the device name actually is, it finds devices and uses them. I also disagree about "much more inconvenience". Please elaborate.
Well, ok, this feature is not concern those who are using only NetworkManager and nothing more. But, quite big portion of people in Fedora are experimenting or just going to get some experience. This feature, IMHO, will make learning threshold higher for them as it will invalidate many of manuals found in the Internet for networking things. For example, creating virtual machines in libvirt with bridging (NM still doesn't support this, AFAIK), firewalling/traffic control/routing, or just binding network daemon to specific interface, etc. For admins and people who help on chats or forums this is also true, but they have enough experience to grasp what is going on. --Atorkhov 15:51, 31 January 2011 (UTC)
Why make an exception for VMs/Guests?
The docs say you will continue to use the ethX naming convention when running as a VM guest. Why? The OS should should exhibit consistant behavior, whether virtualized or not. VMs have virtual hardware that needs to be matched up to detected OS devices in just the same way physical systems do. And how would you reliably "know" that the OS is virtualized?
It's common to write scripts that work across both virtualized and non-virtualized systems. If you continue to use the ethX on virtual, but the new convention on physical, automating NIC changes will require adding a virtualization check and split paths for either case. It also complicates P2V server migrations.
This seems like an ill-conceived change, but if you're going to do it, please consider keeping parity between virtualized and non-virtualized systems.
2011-02-23: one can detect running in a VM with a simple cpuid instruction. However, each VM platform (KVM, Xen, ESX, Hyper-V, ...) exposes its NICs in completely different manners (some are in slot 2, some in slot 3, some embedded, some with no information at all; and you can have all of these in a single VM as well). At least with this change, biosdevname doesn't behave differently for each platform; it just does nothing.
P2V migrations change their MAC addresses during conversion already, so you must be able to account for such changes with the current udev name<->MAC mapping. You can continue using name<->MAC mapping to get names you want, in which case biosdevname won't be invoked.
2011-04-29: If "each VM platform (KVM, Xen, ESX, Hyper-V, ...) exposes its NICs in completely different manners" is the reasoning, how is this any different than saying that every motherboard exposes its NICs in completely different manners? This sounds like a great argument against "consistent network device naming" really being consistent.
2011-04-29: VM BIOSes have so far not exposed the information via either the ACPI _DSM recently added to the spec, or to SMBIOS 2.6 fields. They were only exposing it (if at all) via the PCI IRQ Routing Table, and then, in a highly inconsistent manner. Furthermore, biosdevname is about physicality of a device location, which has no meaning in a VM. It's not about logical "what is this NIC connected to", which is really what you want for a VM. I think it would be cool to have a way for a VM to know which vSwitch or bridge a vNIC is connected to, and to name accordingly, but that is outside the scope of biosdevname. Solutions to that problem are welcome.
2012-02-06: There does not appear to be an exception in the case of VirtualBox. I just installed Fedora 16 and it shows the interface as p2p1.