From Fedora Project Wiki

< FWN‎ | Beats

Line 11: Line 11:
This section contains the discussion happening on the [https://www.redhat.com/mailman/listinfo/fedora-xen fedora-xen list].
This section contains the discussion happening on the [https://www.redhat.com/mailman/listinfo/fedora-xen fedora-xen list].


==== Status of dom0 Support in Vanilla Kernel ====
==== Status of dom0 Support in Upstream Kernel ====
[[PasiKärkkäinen|Pasi Kärkkäinen]] forwarded[1] a message[2] from [[Jeremy Fitzhardinge|Jeremy Fitzhardinge]] originally to the @xen-devel list.
[[PasiKärkkäinen|Pasi Kärkkäinen]] forwarded[1] a message[2] from [[Jeremy Fitzhardinge|Jeremy Fitzhardinge]], originally to the @xen-devel list, describing the state of dom0 support in the upstream <code>kernel</code>.


<pre>
".28 was a bit optimistic; (FWN#137[3]) .29 seems reasonable.  The current dom0 kernel  
.28 was a bit optimistic; .29 seems reasonable.  The current dom0 kernel  
patches can boot up to a fully functional dom0 usersmode, and you can  
patches can boot up to a fully functional dom0 usersmode, and you can  
start xend to see that domain 0 is running.  I *think* in theory you can  
start xend to see that domain 0 is running.  I *think* in theory you can  
create a deviceless domain, but I haven't tried it.  I'm currently  
create a deviceless domain, but I haven't tried it.  I'm currently  
working on blktap support.
working on <code>blktap</code> support.


I really need to put together a proper status update.  Now that dom0  
I really need to put together a proper status update.  Now that dom0  
usermode is working, its a much better base for other people start  
usermode is working, its a much better base for other people start  
contributing.
contributing."
</pre>


[1] http://www.redhat.com/archives/fedora-xen/2008-November/msg00011.html
[1] http://www.redhat.com/archives/fedora-xen/2008-November/msg00011.html
Line 30: Line 28:
[2] http://lists.xensource.com/archives/html/xen-devel/2008-11/msg00205.html
[2] http://lists.xensource.com/archives/html/xen-devel/2008-11/msg00205.html


Just two days later Jeremy posted[3] a large set of patches to @xen-devel.
[3] http://fedoraproject.org/wiki/FWN/Issue137#State_of_Xen_in_Upstream_Linux


<pre>
Just two days later Jeremy posted[4] a large set of patches to @xen-devel with
Here's the chunk of patches to add Xen Dom0 support (it's probably
the following explaination.
worth creating a new xen/dom0 topic branch for it).


A dom0 Xen domain is basically the same as a normal domU domain, but
"A dom0 Xen domain is basically the same as a normal domU domain, but
it has extra privileges to directly access hardware.  There are two
it has extra privileges to directly access hardware.  There are two
issues to deal with:
issues to deal with:
- translating to and from the domain's pseudo-physical addresses and
* translating to and from the domain's pseudo-physical addresses and real machine addresses (for ioremap and setting up DMA)
  real machine addresses (for ioremap and setting up DMA)
* routing hardware interrupts into the domain
- routing hardware interrupts into the domain


ioremap is relatively easy to deal with. An earlier patch introduced
ioremap is relatively easy to deal with. ..."
the _PAGE_IOMAP pte flag, which we use to distinguish between a
regular pseudo-physical mapping and a real machine mapping.
Everything falls out pretty cleanly. A consequence is that the
various pieces of table-parsing code - DMI, ACPI, MP, etc - work out
of the box.


Similarly, the series adds hooks into swiotlb so that architectures
"... Interrupts are a very different affair.  The descriptions in each
can allocate the swiotlb memory appropriately; on the x86/xen side,
Xen hooks these allocation functions to make special hypercalls to
guarantee that the allocated memory is contiguous in machine memory.
 
Interrupts are a very different affair.  The descriptions in each
patch describe how it all fits together in detail, but the overview
patch describe how it all fits together in detail, but the overview
is:
is:


1. Xen owns the local APICs; the dom0 kernel controls the IO APICs
# Xen owns the local APICs; the dom0 kernel controls the IO APICs
2. Hardware interrupts are delivered on event channels like everything else
# Hardware interrupts are delivered on event channels like everything else
3. To set this up, we intercept at pcibios_enable_irq:
# To set this up, we intercept at pcibios_enable_irq:
  - given a dev+pin, we use ACPI to get a gsi
:* given a dev+pin, we use ACPI to get a gsi
  - hook acpi_register_gsi to call xen_register_gsi, which
:* hook acpi_register_gsi to call xen_register_gsi, which
  - allocates an irq (generally not 1:1 with the gsi)
:* allocates an irq (generally not 1:1 with the gsi)
  - asks Xen for a vector and event channel for the irq
:* asks Xen for a vector and event channel for the irq
  - program the IO APIC to deliver the hardware interrupt to the
:* program the IO APIC to deliver the hardware interrupt to the allocated vector
    allocated vector


The upshot is that the device driver gets an irq, and when the
The upshot is that the device driver gets an irq, and when the
Line 85: Line 70:
when running under Xen.
when running under Xen.


TODO: work out how to fit MSI into all this.
TODO: work out how to fit MSI[5] into all this.


So, in summary, this series contains:
'''So, in summary, this series contains:'''
- dom0 console support
* dom0 console support
- dom0 xenbus support
* dom0 xenbus support
- CPU features and IO access for a privleged domain
* CPU features and IO access for a privleged domain
- mtrrs
* mtrrs
- making ioremap work on machine addresses
* making ioremap work on machine addresses
- swiotlb allocation hooks
* swiotlb allocation hooks
- interrupts
* interrupts:
  - introduce PV io_apic operations
** introduce PV io_apic operations
  - add Xen-specific IRQ allocator
** add Xen-specific IRQ allocator
  - switch to using all-Xen event delivery
** switch to using all-Xen event delivery
  - add pirq Xen interrupt type
** add pirq Xen interrupt type
  - table parsing and setup
** table parsing and setup
  - intercept driver interrupt registration
** intercept driver interrupt registration


All this code will compile away to nothing when CONFIG_XEN_DOM0 is not
All this code will compile away to nothing when <code>CONFIG_XEN_DOM0</code> is not
enabled.  If it is enabled, it will only have an effect if booted as a
enabled.  If it is enabled, it will only have an effect if booted as a
dom0 kernel; normal native execution and domU execution should be
dom0 kernel; normal native execution and domU execution should be
unaffected.
unaffected."
</pre>
 
[4] http://lists.xensource.com/archives/html/xen-devel/2008-11/msg00268.html


[3] http://lists.xensource.com/archives/html/xen-devel/2008-11/msg00268.html
[5] http://lwn.net/Articles/44139/


=== Libvirt List ===
=== Libvirt List ===

Revision as of 18:32, 16 November 2008

Virtualization

In this section, we cover discussion on the @et-mgmnt-tools-list, @fedora-xen-list, @libvirt-list and @ovirt-devel-list of Fedora virtualization technologies.

Contributing Writer: Dale Bewley

Enterprise Management Tools List

This section contains the discussion happening on the et-mgmt-tools list

Fedora Xen List

This section contains the discussion happening on the fedora-xen list.

Status of dom0 Support in Upstream Kernel

Pasi Kärkkäinen forwarded[1] a message[2] from Jeremy Fitzhardinge, originally to the @xen-devel list, describing the state of dom0 support in the upstream kernel.

".28 was a bit optimistic; (FWN#137[3]) .29 seems reasonable. The current dom0 kernel patches can boot up to a fully functional dom0 usersmode, and you can start xend to see that domain 0 is running. I *think* in theory you can create a deviceless domain, but I haven't tried it. I'm currently working on blktap support.

I really need to put together a proper status update. Now that dom0 usermode is working, its a much better base for other people start contributing."

[1] http://www.redhat.com/archives/fedora-xen/2008-November/msg00011.html

[2] http://lists.xensource.com/archives/html/xen-devel/2008-11/msg00205.html

[3] http://fedoraproject.org/wiki/FWN/Issue137#State_of_Xen_in_Upstream_Linux

Just two days later Jeremy posted[4] a large set of patches to @xen-devel with the following explaination.

"A dom0 Xen domain is basically the same as a normal domU domain, but it has extra privileges to directly access hardware. There are two issues to deal with:

  • translating to and from the domain's pseudo-physical addresses and real machine addresses (for ioremap and setting up DMA)
  • routing hardware interrupts into the domain

ioremap is relatively easy to deal with. ..."

"... Interrupts are a very different affair. The descriptions in each patch describe how it all fits together in detail, but the overview is:

  1. Xen owns the local APICs; the dom0 kernel controls the IO APICs
  2. Hardware interrupts are delivered on event channels like everything else
  3. To set this up, we intercept at pcibios_enable_irq:
  • given a dev+pin, we use ACPI to get a gsi
  • hook acpi_register_gsi to call xen_register_gsi, which
  • allocates an irq (generally not 1:1 with the gsi)
  • asks Xen for a vector and event channel for the irq
  • program the IO APIC to deliver the hardware interrupt to the allocated vector

The upshot is that the device driver gets an irq, and when the hardware raises an interrupt, it gets delivered on that irq.

We maintain our own irq allocation space, since the hardware-bound event channel irqs are intermixed with all the other normal Xen event channel irqs (inter-domain, timers, IPIs, etc). For compatibility the irqs 0-15 are reserved for legacy device interrupts, but the rest of the range is dynamically allocated.

Initialization also requires care. The dom0 kernel parses the ACPI tables as usual, in order to discover the local and IO APICs, and all the rest of the ACPI-provided data the kernel requires. However, because the kernel doesn't own the local APICs and can't directly map the IO APICs, we must be sure to avoid actually touching the hardware when running under Xen.

TODO: work out how to fit MSI[5] into all this.

So, in summary, this series contains:

  • dom0 console support
  • dom0 xenbus support
  • CPU features and IO access for a privleged domain
  • mtrrs
  • making ioremap work on machine addresses
  • swiotlb allocation hooks
  • interrupts:
    • introduce PV io_apic operations
    • add Xen-specific IRQ allocator
    • switch to using all-Xen event delivery
    • add pirq Xen interrupt type
    • table parsing and setup
    • intercept driver interrupt registration

All this code will compile away to nothing when CONFIG_XEN_DOM0 is not enabled. If it is enabled, it will only have an effect if booted as a dom0 kernel; normal native execution and domU execution should be unaffected."

[4] http://lists.xensource.com/archives/html/xen-devel/2008-11/msg00268.html

[5] http://lwn.net/Articles/44139/

Libvirt List

This section contains the discussion happening on the libvir-list.

OpenVZ bridge support

This is an update of the patch 

http://www.redhat.com/archives/libvir-list/2008-October/msg00326.html

To enable bridge support in the OpenVZ driver. As well as the fixes
suggested last time, it includes an initial bit of HTML doc for the
openvz driver, covering example XML, and the bridge configuration
requirements

[1] http://www.redhat.com/archives/libvir-list/2008-November/msg00117.html

oVirt Devel List

This section contains the discussion happening on the ovirt-devel list.