From Fedora Project Wiki
(AGREED: FESCo Rejects F27 System Wide Change: Rsyslog log format change (+1:5, +0:0, -1:0))
 
(44 intermediate revisions by 3 users not shown)
Line 21: Line 21:


<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
= Change Proposal Name <!-- The name of your change proposal --> =
= Rsyslog log format change proposal <!-- The name of your change proposal --> =


== Summary ==
== Summary ==
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. -->
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. -->
Currently Fedora uses '''RSYSLOG_TraditionalFileFormat''' as a default format for timestamps in its logs. There is missing year and timezone. This proposal aims to change this by adopting ISO 8601 and RFC 3339 compliant timestamp format known as '''RSYSLOG_FileFormat''' instead of current '''RSYSLOG_TraditionalFileFormat'''.


== Owner ==
== Owner ==
<!--  
<!--  
For change proposals to qualify as self-contained, owners of all affected packages need to be included here. Alternatively, a SIG can be listed as an owner if it owns all affected packages.  
For change proposals to qualify as self-contained, owners of all affected packages need to be included here. Alternatively, a SIG can be listed as an owner if it owns all affected packages.  
Line 33: Line 35:
* Name: [[User:rsroka| Radovan Sroka]]
* Name: [[User:rsroka| Radovan Sroka]]
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
* Email: rsroka@redhat.com <your email address so we can contact you, invite you to meetings, etc.>
* Email: rsroka@redhat.com
<!--- * Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
-->
 
* Name: [[User:rpavelka| Roman Pavelka]]
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
* Email: rpavelka@redhat.com
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
Line 44: Line 54:


== Current status ==
== Current status ==
* Targeted release: [[Releases/<number> | Fedora <number> ]]  
* Targeted release: [[Releases/27 | Fedora 27 ]]  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
Line 54: Line 64:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1429009 #1429009]


== Detailed Description ==
== Detailed Description ==


<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
Currently Fedora, RHEL and CentOS use '''RSYSLOG_TraditionalFileFormat''' for log’s timestamp, so timestamps in files like /var/log/messages, /var/log/cron and /var/log/secure looks like e.g.:
<pre>May 29 13:37:50 localhost systemd: Starting Fingerprint Authentication Daemon...</pre>
This format has few disadvantages
* Does not include year which sometimes may be needed, mostly when doing long term analysis or some investigation.
* Does not include timezone which may be important piece when working with system scattered around the globe.
* It is not standard format. Standards are ISO 8601 and more strict RFC 3339
We would propose to change this to defaults to standard format with timezone included. We are suggesting '''RSYSLOG_FileFormat''' that looks like e.g.:
<pre>2017-05-29T13:40:50.976409+02:00 localhost systemd: Stopping System Logging Service...</pre>
This also matches short-iso-precise output format in journalctl added in the upcoming systemd v234. All necessary changes are:
<pre>$ diff rsyslog.conf.orig rsyslog.conf -u
--- rsyslog.conf.orig    2017-05-26 16:31:09.768671818 +0200
+++ rsyslog.conf    2017-05-26 16:30:36.093682748 +0200
@@ -30,7 +30,7 @@
global(workDirectory="/var/lib/rsyslog")
# Use default timestamp format
-module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")
+module(load="builtin:omfile" Template="RSYSLOG_FileFormat")
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf</pre>


== Benefit to Fedora ==
== Benefit to Fedora ==
Line 65: Line 106:
    
    
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?-->
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?-->
Compliance to international standard, more information available in logs.


== Scope ==
== Scope ==
* Proposal owners:
* Proposal owners:
** commit necessary changes
** create rsyslog build
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
 
* Other developers: none <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Other developers: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
 
* Release engineering: [https://pagure.io/releng/issue/6818 Releng#6818]
* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engeneering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
<!-- [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engeneering is needed) --> <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuid required?  include a link to the releng issue.  
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuid required?  include a link to the releng issue.  
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
 
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: Not affected <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
 
* Policies and guidelines: Not affected <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Policies and guidelines: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
 
* Trademark approval: Not needed for this Change
* Trademark approval: N/A (not needed for this Change)
<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://fedorahosted.org/council/ ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->
<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://fedorahosted.org/council/ ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->


Line 90: Line 131:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)
Someone parsing logs would need update his/her scripts or revert configuration to traditional one.


== How To Test ==
== How To Test ==
Line 108: Line 149:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)
Restart rsyslog by “systemctl restart rsyslog” and check “sudo tail /var/log/messages”, timestamp should be in ISO 8601 format, e.g.:
2017-05-26T16:25:47.125900+02:00


== User Experience ==
== User Experience ==
<!-- If this change proposal is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
<!-- If this change proposal is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)
Regular users should not experience any change. Administrators would benefit from more information provided, especially when maintaining a system scattered in more timezones or doing long-term analysis.


== Dependencies ==
== Dependencies ==
Line 119: Line 161:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)
None


== Contingency Plan ==
== Contingency Plan ==


<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
* Contingency mechanism: Rsyslog maintainer will either change the configuration or keep the original.
* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Contingency deadline: Fedora 27 Beta freeze
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
* Blocks release? No
* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Blocks product? No
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Blocks product? product <!-- Applicable for Changes that blocks specific product release/Fedora.next -->


== Documentation ==
== Documentation ==
Line 135: Line 174:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)
http://www.rsyslog.com/doc/v8-stable/configuration/templates.html


== Release Notes ==
== Release Notes ==
Line 151: Line 190:


<!-- Select proper category, default is Self Contained Change -->
<!-- Select proper category, default is Self Contained Change -->
[[Category:SelfContainedChange]]
<!-- [[Category:SelfContainedChange]] -->
<!-- [[Category:SystemWideChange]] -->
[[Category:SystemWideChange]]

Latest revision as of 13:09, 17 July 2017


Rsyslog log format change proposal

Summary

Currently Fedora uses RSYSLOG_TraditionalFileFormat as a default format for timestamps in its logs. There is missing year and timezone. This proposal aims to change this by adopting ISO 8601 and RFC 3339 compliant timestamp format known as RSYSLOG_FileFormat instead of current RSYSLOG_TraditionalFileFormat.

Owner

Current status

Detailed Description

Currently Fedora, RHEL and CentOS use RSYSLOG_TraditionalFileFormat for log’s timestamp, so timestamps in files like /var/log/messages, /var/log/cron and /var/log/secure looks like e.g.:

May 29 13:37:50 localhost systemd: Starting Fingerprint Authentication Daemon...

This format has few disadvantages

  • Does not include year which sometimes may be needed, mostly when doing long term analysis or some investigation.
  • Does not include timezone which may be important piece when working with system scattered around the globe.
  • It is not standard format. Standards are ISO 8601 and more strict RFC 3339

We would propose to change this to defaults to standard format with timezone included. We are suggesting RSYSLOG_FileFormat that looks like e.g.:

2017-05-29T13:40:50.976409+02:00 localhost systemd: Stopping System Logging Service...

This also matches short-iso-precise output format in journalctl added in the upcoming systemd v234. All necessary changes are:

$ diff rsyslog.conf.orig rsyslog.conf -u
--- rsyslog.conf.orig    2017-05-26 16:31:09.768671818 +0200
+++ rsyslog.conf    2017-05-26 16:30:36.093682748 +0200
@@ -30,7 +30,7 @@
 global(workDirectory="/var/lib/rsyslog")
 
 # Use default timestamp format
-module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")
+module(load="builtin:omfile" Template="RSYSLOG_FileFormat")
 
 # Include all config files in /etc/rsyslog.d/
 $IncludeConfig /etc/rsyslog.d/*.conf

Benefit to Fedora

Compliance to international standard, more information available in logs.

Scope

  • Proposal owners:
    • commit necessary changes
    • create rsyslog build
  • Other developers: none
  • Release engineering: Releng#6818
  • Policies and guidelines: Not affected
  • Trademark approval: Not needed for this Change

Upgrade/compatibility impact

Someone parsing logs would need update his/her scripts or revert configuration to traditional one.

How To Test

Restart rsyslog by “systemctl restart rsyslog” and check “sudo tail /var/log/messages”, timestamp should be in ISO 8601 format, e.g.: 2017-05-26T16:25:47.125900+02:00

User Experience

Regular users should not experience any change. Administrators would benefit from more information provided, especially when maintaining a system scattered in more timezones or doing long-term analysis.

Dependencies

None

Contingency Plan

  • Contingency mechanism: Rsyslog maintainer will either change the configuration or keep the original.
  • Contingency deadline: Fedora 27 Beta freeze
  • Blocks release? No
  • Blocks product? No

Documentation

http://www.rsyslog.com/doc/v8-stable/configuration/templates.html

Release Notes