From Fedora Project Wiki
(Created page with "{{ Admon/warning | In progress. Any help appreciated. | This is a start page for a detailed Websites team presentation. You can edit freely.}} __TOC__ == The websites arch ...")
 
m (Minor change)
 
(52 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{{ Admon/warning | In progress. Any help appreciated. | This is a start page for a detailed Websites team presentation. You can edit freely.}}
{{autolang|base=yes}}
{{header|websites}}


__TOC__




== The websites arch (build, dev and trans) ==
== The websites arch (build, dev and trans) ==
=== Our build system ===
=== Our build system ===
Our websites are rebuilt hourly. This is done using the syncStatic script which is puppet managed. In other words, you need to be on the infra team to change it. A sample copy is uploaded [http://shaiton.fedorapeople.org/web/syncStatic.sh there]. This is where we decide against which branch we build each websites. There is also a specific syncStatic.stg script for the staging websites. Example of stg websites are stg.fedoraproject.org and spins.stg.fedoraproject.org.  
Our websites are rebuilt hourly. This is done using the syncStatic script which is puppet managed. In other words, you need to be on the infra team to change it. A sample copy is uploaded [http://shaiton.fedorapeople.org/web/syncStatic.sh here]. This is where we decide against which branch we build each websites. There is also a specific syncStatic.stg script for the staging websites. Example of stg websites are stg.fedoraproject.org and spins.stg.fedoraproject.org.  


For release time, please refer to our specific documentation [http://infrastructure.fedoraproject.org/infra/docs/fedorawebsites.txt there].
For release time, please refer to our [http://infrastructure.fedoraproject.org/infra/docs/fedorawebsites.rst specific documentation]. We have also a document which helps you [https://infrastructure.fedoraproject.org/infra/docs/fedorawebsites.rst not to forget anything] when merging to the next release version.
 
==== Usage ====
To build the static websites pages you can run ''make'' from the specific website root directory. This will build the pages in all available languages. A list of all existing translations is available in the header of the Makefile.<br />
This kind of building is rather long, if you want you can run also:
* ''make LANG'': builds the pages in a specific language, where LANG is the two-letter abbreviation for the translation.
* ''make LANG test'': builds a test instance at http://localhost:5000/ in a specific language, run '''make stoptest''' to take it down.
* ''make test'': same as above, will build a test instance for all available languages.


=== Coding ===
=== Coding ===
''insert graph here''
Many of our websites are using the [http://getbootstrap.com/css/ Bootstrap System] which is why you will see many divs in our HTML source files. You might want to see an [http://getbootstrap.com/getting-started/ explanation].
 
==== Architecture ====
==== Architecture ====
There is a specific folder for each websites under which you will find the following tree:
The fedora-web repository looks like the following (mostly all websites use the same tree as boot.fedoraproject.org).


    .
  .
    |-- boot.fedoraproject.org
  |-- boot.fedoraproject.org
    |  |-- build
  |-- data
    |  |-- ChangeLog
  |  |-- Makefile
    |  |-- data
  |  |-- po
    |  |-- httpd
  |  `-- static
    |  |-- Makefile
  |-- build.d
    |  |-- po
  |  |-- buildconf.py
    |  `-- static
  |  |-- build.py
    `-- TODO
  |  |-- construct-translations.py
  |  |-- globalvar.py
  |  |-- pybabel.cfg
  |  `-- translations.ini
  …
  |-- httpd
  |-- Makefile.in
  `-- tools


There are several folders, one for each websites. The main one have the same tree: a build folder where we have some python scripts for Genshi, the Makefile which is used to run the build, po which hold the translation files, data who has some templates and the HTML sources (in data/content/), and static where are some static files (pictures, JS scripts, CSS..).
Each websites directory consists on a <code>data</code> folders holding HTML code, while all static code (JS, pictures, CSS…) are in <code>static</code>. Some websites use specific building files in the <code>build</code> dir if any.
The httpd folder is used to set a local apache server in order to run a testing instance.
<code>build.d</code> contains the python building scripts and the Genshi template, as well as the language name localized for each luangages.
The <code>httpd</code> folder contains standalone apache files to locally test our websites.
<code>tools</code> is the place for manual script that we run times to time. In order to update the <code>LINGUAS</code> file, or to check the websites..


==== i18n ====
==== i18n ====
Line 36: Line 53:
# <code>${_('String')}</code>
# <code>${_('String')}</code>
# <code>${Markup(_('String with &amp;gt; html code'))}</code>
# <code>${Markup(_('String with &amp;gt; html code'))}</code>
# <code>${Markup(_('String with a &amp;lt;a href="%s"&amp;gt;link&amp;lt;/a&amp;gt;') % 'http://fedoraproject.org')}</code>
# <code><nowiki>${Markup(_('String with a &amp;lt;a href="%s"&amp;gt;link&amp;lt;/a&amp;gt;') % 'http://fedoraproject.org')}</nowiki></code>
# <code>${_('%(size)s, DVD ISO disc image for %(arch)s-bit PC') % {'size':'3.1 GB', 'arch':'32'}}</code>
# <code>${_('%(size)s, DVD ISO disc image for %(arch)s-bit PC') % {'size':'3.1 GB', 'arch':'32'}}</code>
# <code><nowiki>${Markup(_('If you think you\'ve found a bug, read &lt;a href="%s"&gt;here&lt;/a&gt; fist.') % ('http://fedoraproject.org/wiki/Common_F'+global_variables.release['prev_id']+'_bugs'))}</nowiki></code>


Fist you have the simplest example. Then the markup that could be used everywhere (it works with embedded html code, not as the first one). Then, you have an example using a variable. That is really useful in order to avoid having translators to translate again the string when changing the variable. The last example is a most complete one which shows you how to deal with multiple variables. The translator will be able to change the variable order using it's name.
Fist you have the simplest example. Then the markup that could be used everywhere (it works with embedded html code, not as the first one). Then, you have an example using a variable. That is really useful in order to avoid having translators to translate again the string when changing the variable. The last example is a most complete one which shows you how to deal with multiple variables. The translator will be able to change the variable order using it's name.
Line 45: Line 63:
Once it's correct, you need to generate the new translation template file (POT) with <code>make pot</code>.
Once it's correct, you need to generate the new translation template file (POT) with <code>make pot</code>.


=== l10n ===
For general information about Genshi templates, see:
All POs are download hourly. Any error should be reported to our [https://fedorahosted.org/fedora-websites/newticket tracking system].
* http://genshi.edgewall.org/wiki/Documentation, and in particular,
* http://genshi.edgewall.org/wiki/Documentation/xml-templates.html


Whenever the HTML content change, we need to update the POT as explained on the previous section. Then, we need to push this POT to the translation platform, [http://fedora.transifex.com/ transifex.com]. This is done by the command <code>make pushpot</code>. But only the maintainers of the [https://fedora.transifex.com/projects/p/fedora-web/ fedora-web] release are able to push a new POT. Ask them if you need to do so. Please, open a new ticket.
===== Variables =====
As described in the examples above, you will note there are a lot of variables instead of hard codes. These variables have been set to make it easier to update our webpages. When merging from one branch to the next, i.e. from beta to final release, all the content of our pages can be updated by editing simply one single file, where all the variables are defined.<br />
This file actually is used in the sites of ''fedoraproject.org'' and in ''spins.fedoraproject.org'' and is located in <code>~/build.d/globalvar.py</code>


=== l10n ===
All POs are download twice a day. Any error should be reported to our [https://fedorahosted.org/fedora-websites/newticket tracking system].
We download only the files translated at least at 10%. Please note that you won't be able to proofread only if you didn't get so far. This value is in the [http://git.fedorahosted.org/cgit/fedora-web.git/tree/Makefile.in#n66 Makefile.in] for the local test and in puppet/syncTranslation.sh for the online websites.


If you need to add a new language, you need to update the LINGUAS file, the translation.py file, and the languages.conf file.
Whenever the HTML content change, we need to update the POT as explained on the previous section. Then, we need to push this POT to the translation platform, [http://fedora.zanata.com/ zanata.org]. This is done by the command <code>make pushpot</code>. But only the maintainers of the [https://fedora.zanata.org/project/view/fedora-web/ fedora-web] release are able to push a new POT. Ask them if you need to do so opening a new ticket or ask directly on IRC.
 
Here is the tree:
    .
    |-- boot.fedoraproject.org
    |  |-- build
    |  |      `-- translations.py
    |  |-- httpd
    |  |      `-- conf
    |  |              `-- languages.conf.in
    |  `-- po
    |        `--LINGUAS
    `-- l10N_update.sh
 
The script "l10N_update.sh" could be used to generate a new LINGUAS file as well as to upload a new POT and to backup the PO files. There is a redesign on going in order to automatize this, and have only one translation.py and languages.conf.in files. This is ticket [https://fedorahosted.org/fedora-websites/ticket/90 #90].
 
== The Hot (comming) New Websites Arch ==
 
The idea is to have a different tree using a recursive make. The syncStatic script will have few changes. Like the following (not sure about the httpd folder)
 
  .
  |-- boot.fedoraproject.org
  |  |-- build
  |  |-- data
  |  |-- httpd
  |  |-- Makefile
  |  |-- po
  |  `-- static
  …
  |-- build
  |-- Makefile
  |-- static
  |-- templates
  `-- TODO
 
 
This is directly related to ticket [https://fedorahosted.org/fedora-websites/ticket/90 #90]. To help, get in touch with shaiton.
 
== How to join/help ==
Joining the websites team is not very difficult and doesn't differ too much from the join procedure of the other Sub-Projects.<br />
Obviously the best starting point is the [https://fedoraproject.org/wiki/Websites/Join join page], where you should follow the points step by step.
PHP and Python skills are welcome, because our sites are build with HTML/CSS and the Genshi framework
for internationalization as mentioned above.<br />
It would be a good idea taking a look at the [http://960.gs/ 960 grid system], because there are many ''divs'' in our
sources.<br />
First of all join the [https://admin.fedoraproject.org/mailman/listinfo/websites websites mailing list] and send a self introduction.
 
To get started, you need also:
 
=== Clone the websites repository ===
 
* Ensure you have the {{package|git}} package and the Apache web server installed:
<pre>su -c 'yum shell'
> install git gettext python-genshi python-setuptools babel python-feedparser
> groupinstall 'Web Server'
> run</pre>
 
* Set up at least your name and email address in your global ''git'' configuration:
git config --global user.name "John Smith"
git config --global user.email "john.smith@mailbox.com"
git config --global color.ui auto
 
* Make sure you are in a directory to which you have write access, such as your home directory or a subdirectory, and clone the websites repository.  ''This takes several minutes on a fast connection.''
<pre>cd $HOME
git clone git://git.fedorahosted.org/git/fedora-web.git</pre>
 
* Change directory to the website you want to change.  The repository carries several sites, including fedoraproject.org, spins.fedoraproject.org, and others.
<pre>cd fedora-web/fedoraproject.org</pre>
 
* You can make a fresh, new branch for your changes, or you can do your work against an existing branch of the ''fedora-web'' project. To make a fresh branch for your own use only ("mynewbranch" is a name you can choose):
<pre>git checkout -b mynewbranch</pre>
If you want to work on something that already exists as a branch on websites, like a redesign branch, you can set up your branch to track the changes on that branch.  That allows you to build on the existing work without worrying about whether you're overwriting the existing team's work.  For example, if you want to help with the "fpo-redesign" branch:
<pre>git checkout -b mynewbranch --track origin/fpo-redesign</pre>
 
Make your changes.  It's best to make a set of related changes together, but don't make many unrelated changes at once.  For instance, it's OK to make several spelling fixes together, but don't combine those with an additional navigation menu option.
 
Test your changes.
<pre>make</pre>
After the make completes successfully and the website will now be available in your /out/ directory.
Now run a local sandbox test of the website:
<pre>make test</pre>
 
Point your web browser at http://localhost:5000 to view the sandbox site.  When you are done viewing, stop the temporary web server:
<pre>make stoptest</pre>
 
If everything looks OK, stage the files you changed:
<pre>git add data/content/file1.html data/content/file2.html</pre>
 
Commit the staged changes:
<pre>git commit -m 'Fix spelling errors'</pre>
 
=== Sending in your changes ===
 
* When you're done, create patch files for your changes:
<pre>git format-patch origin/master</pre>
 
* Now you can send the patches you've created, which will have names such as ''0001-my-change.patch'', to the {{fplist|websites}} list or upload your patch to our [https://fedorahosted.org/fedora-websites/report/1 track] if you fixed a ticket.
 
=== Join the FAS Group ===
 
Log in at https://admin.fedoraproject.org/accounts/login and join the websites group.


=== Work on tickets ===
In order to know if the builder is actually pulling the translations in a specific branch/website, check on the [http://git.fedorahosted.org/cgit/fedora-web.git/tree/ repo] if a file named <code>PO_FREEZE</code> exists on the website folder. This could be used while editing the Transifex POT without overriding the existing translations on production.


Start working on [https://fedorahosted.org/fedora-websites/report/1 open tickets], take a look also on the [http://fedoraproject.org/easyfix/#fedora-websites easyfix] page.
Adding a new language is done through 4 steps:
# first, create it at zanata.org by uploading a new translation on the website.
# Then, update the <code>translations.ini</code> file accordingly for local builds
# update the <code>LINGUAS</code> file (<code>tools/l10N_update.sh</code> might be of help) - for local builds
# update the local <code>httpd/conf/language.conf</code> file (by running <code>make en test</code> for example) and ask to make this change in the infra (puppet). That is probably needed as we have the same exact copy of this file there. File a ticket for that on our [https://fedorahosted.org/fedora-websites/newticket?type=enhancement&summary=adding%20new%20locale%20to%20puppet&keywords=infra,%20l10n Trac].


=== Join IRC channel and Mailing List ===
== Joining/helping ==
Please read the [[Websites/Join]] and [[How_to_fix_bugs_on_the_Fedora_Project_website|How to fix bugs]] pages.


Please ask if you have any doubts, writing to the [https://admin.fedoraproject.org/mailman/listinfo/websites Websites Mailing List] or on IRC channel #fedora-websites,
== Actual used branches for each (sub)domain ==
located on the freenode server.


=== Request sponsorship to the 'gitfedora-web' group ===
All our websites in production now are built against master branch. This makes it much easier for contributors to push their commits to the right branch. Staging websites on the other hand are built against the actual developing ''release-state'' branch: this could be alpha or beta, but in some cases can also be a specific branch. In fact we use the staging (stg) branch for specific enhancements, this allows us to keep the alpha or beta branch clean.<br />
The new workflow means we will merge every single development branch to master at D-day, as production sites are built only against master branch.<br />
This overview should help you to understand which git-branch is used to build a single subdomain or domain.


After you've done successfully your first changes and sent them as patch for the websites team, you can request sponsorhip to the 'gitfedora-web' group, which will allow you to commit to the [http://git.fedorahosted.org/git/?p=fedora-web.git fedora-web repository] (where the website source code is at).
{|
|- style=" color: #fff; background-color: #3074c2;" tablewidth="98%"
| '''Domain''' ||'''Branch'''
|-
| [http://getfedora.org getfedora.org] || master
|-
| [http://spins.fedoraproject.org spins.fedoraproject.org] || master
|-
| [http://labs.fedoraproject.org labs.fedoraproject.org] || master
|-
| [http://arm.fedoraproject.org arm.fedoraproject.org] || master
|-
| [http://stg.getfedora.org stg.getfedora.org] || f24-beta
|-
| [http://spins.stg.fedoraproject.org spins.stg.fedoraproject.org] || f24-beta
|-
| [http://labs.stg.fedoraproject.org labs.stg.fedoraproject.org] || f24-beta
|-
| [http://arm.stg.fedoraproject.org arm.stg.fedoraproject.org] || f24-beta
|-
| [http://fudcon.fedoraproject.org fudcon.fedoraproject.org] || master
|-
| [http://boot.fedoraproject.org boot.fedoraproject.org] || master
|-
| [http://fedoracommunity.org fedoracommunity.org] || master
|-
| [http://fedorahosted.org fadorahosted.org] || master
|-
| [http://fedorapeople.org fedorapeople.org] || master
|-
| [http://flocktofedora.org flocktofedora.org] || master
|-
| [http://start.fedoraproject.org start.fedoraproject.org] || master
|-
|}
[[category:websites]]

Latest revision as of 02:02, 8 October 2018



The websites arch (build, dev and trans)

Our build system

Our websites are rebuilt hourly. This is done using the syncStatic script which is puppet managed. In other words, you need to be on the infra team to change it. A sample copy is uploaded here. This is where we decide against which branch we build each websites. There is also a specific syncStatic.stg script for the staging websites. Example of stg websites are stg.fedoraproject.org and spins.stg.fedoraproject.org.

For release time, please refer to our specific documentation. We have also a document which helps you not to forget anything when merging to the next release version.

Usage

To build the static websites pages you can run make from the specific website root directory. This will build the pages in all available languages. A list of all existing translations is available in the header of the Makefile.
This kind of building is rather long, if you want you can run also:

  • make LANG: builds the pages in a specific language, where LANG is the two-letter abbreviation for the translation.
  • make LANG test: builds a test instance at http://localhost:5000/ in a specific language, run make stoptest to take it down.
  • make test: same as above, will build a test instance for all available languages.

Coding

Many of our websites are using the Bootstrap System which is why you will see many divs in our HTML source files. You might want to see an explanation.

Architecture

The fedora-web repository looks like the following (mostly all websites use the same tree as boot.fedoraproject.org).

  .
  |-- boot.fedoraproject.org
  |   |-- data
  |   |-- Makefile
  |   |-- po
  |   -- static
  |-- build.d
  |   |-- buildconf.py
  |   |-- build.py
  |   |-- construct-translations.py
  |   |-- globalvar.py
  |   |-- pybabel.cfg
  |   -- translations.ini
  …
  |-- httpd
  |-- Makefile.in
  `-- tools

Each websites directory consists on a data folders holding HTML code, while all static code (JS, pictures, CSS…) are in static. Some websites use specific building files in the build dir if any. build.d contains the python building scripts and the Genshi template, as well as the language name localized for each luangages. The httpd folder contains standalone apache files to locally test our websites. tools is the place for manual script that we run times to time. In order to update the LINGUAS file, or to check the websites..

i18n

We use the Genshi framework for internationalization. Our current templates need to have special markup around the strings in order to mark them as translatable.

There are several way to use the markups.

  1. ${_('String')}
  2. ${Markup(_('String with &gt; html code'))}
  3. ${Markup(_('String with a &lt;a href="%s"&gt;link&lt;/a&gt;') % 'http://fedoraproject.org')}
  4. ${_('%(size)s, DVD ISO disc image for %(arch)s-bit PC') % {'size':'3.1 GB', 'arch':'32'}}
  5. ${Markup(_('If you think you\'ve found a bug, read <a href="%s">here</a> fist.') % ('http://fedoraproject.org/wiki/Common_F'+global_variables.release['prev_id']+'_bugs'))}

Fist you have the simplest example. Then the markup that could be used everywhere (it works with embedded html code, not as the first one). Then, you have an example using a variable. That is really useful in order to avoid having translators to translate again the string when changing the variable. The last example is a most complete one which shows you how to deal with multiple variables. The translator will be able to change the variable order using it's name.

To test that you haven't missed anything (for example you need to escape the simple quote from the string), try with make en test in order to build the website for the en language.

Once it's correct, you need to generate the new translation template file (POT) with make pot.

For general information about Genshi templates, see:

Variables

As described in the examples above, you will note there are a lot of variables instead of hard codes. These variables have been set to make it easier to update our webpages. When merging from one branch to the next, i.e. from beta to final release, all the content of our pages can be updated by editing simply one single file, where all the variables are defined.
This file actually is used in the sites of fedoraproject.org and in spins.fedoraproject.org and is located in ~/build.d/globalvar.py

l10n

All POs are download twice a day. Any error should be reported to our tracking system. We download only the files translated at least at 10%. Please note that you won't be able to proofread only if you didn't get so far. This value is in the Makefile.in for the local test and in puppet/syncTranslation.sh for the online websites.

Whenever the HTML content change, we need to update the POT as explained on the previous section. Then, we need to push this POT to the translation platform, zanata.org. This is done by the command make pushpot. But only the maintainers of the fedora-web release are able to push a new POT. Ask them if you need to do so opening a new ticket or ask directly on IRC.

In order to know if the builder is actually pulling the translations in a specific branch/website, check on the repo if a file named PO_FREEZE exists on the website folder. This could be used while editing the Transifex POT without overriding the existing translations on production.

Adding a new language is done through 4 steps:

  1. first, create it at zanata.org by uploading a new translation on the website.
  2. Then, update the translations.ini file accordingly for local builds
  3. update the LINGUAS file (tools/l10N_update.sh might be of help) - for local builds
  4. update the local httpd/conf/language.conf file (by running make en test for example) and ask to make this change in the infra (puppet). That is probably needed as we have the same exact copy of this file there. File a ticket for that on our Trac.

Joining/helping

Please read the Websites/Join and How to fix bugs pages.

Actual used branches for each (sub)domain

All our websites in production now are built against master branch. This makes it much easier for contributors to push their commits to the right branch. Staging websites on the other hand are built against the actual developing release-state branch: this could be alpha or beta, but in some cases can also be a specific branch. In fact we use the staging (stg) branch for specific enhancements, this allows us to keep the alpha or beta branch clean.
The new workflow means we will merge every single development branch to master at D-day, as production sites are built only against master branch.
This overview should help you to understand which git-branch is used to build a single subdomain or domain.

Domain Branch
getfedora.org master
spins.fedoraproject.org master
labs.fedoraproject.org master
arm.fedoraproject.org master
stg.getfedora.org f24-beta
spins.stg.fedoraproject.org f24-beta
labs.stg.fedoraproject.org f24-beta
arm.stg.fedoraproject.org f24-beta
fudcon.fedoraproject.org master
boot.fedoraproject.org master
fedoracommunity.org master
fadorahosted.org master
fedorapeople.org master
flocktofedora.org master
start.fedoraproject.org master