The login programs and/or pam have SELinux awareness built into them.
gdm and xdm have SELinux awareness built into them, while login and ssh use the pam_selinux. In RHEL4 su and sudo also used pam_selinux. The way this works is the user authenticates to the system using whatever authorization mechanism is used. The the login programs call into selinux functions to setup the default session.
In FC5 the SELinux login code searches /etc/selinux/POLICYTYPE/seusers file for the Linux username. If it finds a match it returns the second field as the "SELinux Name", it also grabs the third field as the range of MLS/MCS Security Categories. Login uses __default__ when it does not find a match.
Next the SELinux Login calls a function which returns the default login context for this SELinux User. In RHEL4 the login name would need to match the SELinux Name or you would default to user_u. This functions tries to read the file /etc/selinux/POLICYTYPE/contexts/users/SELINUXUSER and if that does not exist it falls back to /etc/selinux/POLICYTYPE/contexts/default_contexts. These files contain a mapping between the context of the "login program" and a list of contexts to login as. The SELinux login code will select the first "reachable" context in this file. So as an example:
If you are logging in via sshd as root on a strict machine, it will search for /etc/selinux/strict/contexts/users/root and then look for the line system_r:remote_login_t. It will then check with the installed policy to check which contexts are reachable, by the login user. When it finds a match it returns the context. This would allow you to set up different context for a user depending on if he logs in via the console, ssh or gdm.
Now the login program asks the kernel for the terminal type to assign to this user. It takes this terminal type and sets it on the tty. Next it tells the kernel the context to set on all execs from this process. At this point the login programs execs the shell and the user is logged in. When the user logs out the default exec context gets reset and the terminal is reset back to the previous settings.
Why are su and sudo no longer using pam_selinux in FC5?
Well these functions were originally added to su and sudo in order to allow the SELinux User identity change at the same time as the Linux Identity. But on a targeted machine this was a noop since you stayed as user_u:system_r:unconfined_t through this process. It was also decided that for MLS and Strict policy this was not a great idea, also with the introduction of MLS, it became difficult to figure out what sensitivity level the user would change to. So on MLS/Strict you now need to use su/sudo to change your Linux Identity and newrole to change your SELinux Role or MLS Level.
What about parameters of pam_selinux?
The "multiple" parameter is used to allow the user to choose which context they are able to log in as. For example on an MLS machine you may be able to login to the root account as staff_t, sysadm_t or secadm_t. So if you added multiple to the pam_selinux it would display all three context and allow you to choose. This parameter has been problematic for targeted policy and should be removed if you have it in a pam module.
The "debug" option will turn on debugging and will add lots of messages to the syslog.
The "open"/"close" options, are interesting. pam_selinux is called on session start and on session exit, and these parameters are used to tell us whether we are opening a session or closing one. We needed to put one pam_selinux at the beginning of the session to make sure everything was cleaned up and one as the last option in the session rule. pam_session open causes the execute security context to be set, so all execs from that point on are called with the security context. The only thing we want called is the exec of the shell. When the session ends we want to reset the session back to the default before any other sessions are run. This prevents another session from accidentally exec-ing a program with the wrong context.
This was a fairly advanced topic and hopefully you understood it. Most of these advanced features are more important for Strict and MLS policy, since targeted usually logs in as unconfined_t.