From Fedora Project Wiki
Attendees
- adamw (77)
- nirik (21)
- tflink (20)
- jreznik (18)
- brunowolff (16)
- kparal (9)
- zodbot (4)
- jsmith (3)
- Martix (2)
- mkrizek (2)
- Cerlyn (1)
- pschindl (1)
Agenda
- Fedora 20 Change validation
- Test Days
- Open floor
Fedora 20 Change validation
- At last week's meeting jreznik requested validation of at least systemwide Changes for F20: can we do this?
- After some discussion, it was agreed that we cannot commit to do any form of 'validation testing' for any and all systemwide Changes sight unseen: when FESCo approves the F20 ChangeSet we will re-consider this issue and come up with a realistic plan for testing what we can of the approved Changes
Test Days
- We were asked by dpal if we can have an 'out of cycle' test day for some planned FreeIPA enhancements to be added to F19 as updates
- We agreed there are no problems with an out of cycle test day, we'll run it as always and append it to the F19 test day schedule
- Fedora 20 schedule may be quite tight so we should send out call for F20 test days soon
Open floor
- wwoods was looking for help triaging fedup bugs, contact nirik to sign up
- nirik will talk to anaconda team about doing regular installer composes (again?)
Action items
N/A
IRC Log
| adamw | #startmeeting Fedora QA meeting | 15:04 |
|---|---|---|
| zodbot | Meeting started Mon Jul 8 15:04:26 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:04 |
| zodbot | Useful Commands: #action #agreed #halp #info #idea #link #topic. | 15:04 |
| adamw | #meetingname fedora-qa | 15:04 |
| zodbot | The meeting name has been set to 'fedora-qa' | 15:04 |
| adamw | #topic Roll call | 15:04 |
| * kparal here | 15:04 | |
| * mkrizek is here | 15:04 | |
| * Cerlyn is here | 15:04 | |
| adamw | sorry for the late start, folks - who's around for funtimes? | 15:04 |
| * tflink is here | 15:05 | |
| * nirik is lurking. | 15:05 | |
| * brunowolff is here | 15:05 | |
| adamw | yay funtimes | 15:06 |
| * jsmith is lurking | 15:07 | |
| adamw | like a ninja | 15:07 |
| jsmith | I'm sneaky like that :-) | 15:07 |
| adamw | alrighty! | 15:07 |
| adamw | well, all i had for the agenda was a couple of things left from last time | 15:08 |
| adamw | #topic Fedora 20 Change validation | 15:08 |
| adamw | #info last week, jreznik asked if it was possible for us to do some form of 'validation testing' for the systemwide Changes for Fedora 20 | 15:08 |
| kparal | which changes? | 15:09 |
| adamw | i figured we'd better discuss it | 15:09 |
| adamw | kparal: Changes as in 'what used to be called Features' | 15:09 |
| kparal | ah | 15:09 |
| adamw | one of the problems is that AFAIK we don't have a list of them all yet | 15:09 |
| kparal | validation after they are deemed complete, or before? | 15:09 |
| nirik | the deadline is the 16th. | 15:10 |
| tflink | and "some form of validation testing" is very vague | 15:10 |
| adamw | well, that's one of the things to discuss, i guess. | 15:10 |
| tflink | nirik: the 16th of july? | 15:11 |
| * jreznik is here | 15:11 | |
| adamw | jreznik: is https://fedoraproject.org/wiki/Category:ChangeReadyForWrangler really all the Changes that are proposed right now? | 15:11 |
| nirik | yes. | 15:11 |
| nirik | https://fedoraproject.org/wiki/Releases/20/Schedule | 15:11 |
| * pschindl is here. | 15:11 | |
| * adamw notes https://fedoraproject.org/wiki/Category:SystemWideChanges is empty | 15:11 | |
| nirik | so, they arent even all in yet. | 15:11 |
| adamw | right, which makes it hard to eyeball a list and come up with an answer | 15:11 |
| jreznik | https://fedoraproject.org/wiki/Category:ChangeAcceptedF20 for accepted | 15:11 |
| adamw | jreznik: did you have a more specific idea in mind for exactly what you'd like us to be doing here? | 15:12 |
| * adamw brb, call of nature | 15:12 | |
| tflink | wow, I didn't realize that the downtime between f19 and f20 was so short | 15:12 |
| adamw | #chair kparal brunowolff | 15:12 |
| zodbot | Current chairs: adamw brunowolff kparal | 15:12 |
| jreznik | https://fedoraproject.org/wiki/Category:SystemWideChange should not be empty (it's not...) | 15:12 |
| adamw | oh, I missed the 's' | 15:13 |
| adamw | er, added an s. | 15:13 |
| jreznik | tflink: it's really short, that's probably the reason why not so many proposals were submitted so far | 15:13 |
| jreznik | and also f18, f19 were big, f20 seems to be smaller | 15:13 |
| kparal | I assumed we're branching in September? | 15:13 |
| brunowolff | August 8th I think. | 15:13 |
| tflink | kparal: nope, august 06 | 15:13 |
| * adamw notes that https://fedoraproject.org/wiki/Releases/20/ChangeSet has the incorrect category names | 15:13 | |
| brunowolff | Alpha freeze is Augist 20th I think. | 15:13 |
| kparal | wow, the world is speeding up | 15:13 |
| * tflink should realy start paying more attention to fesco, this is going to impact automation plans | 15:14 | |
| jreznik | adamw: for ChangeSet - should be ready this week, I need to tweak my old scripts :( | 15:14 |
| jreznik | tflink: the main reason why fesco asked me for this schedule was to avoid christmas holidays (there's approx 1 month buffer now) | 15:15 |
| jreznik | and it's not final schedule - I think FESCo would listen to QA if you would need more time (and I'd say - propose QA automation as Change...) | 15:15 |
| tflink | jreznik: yeah, I understand that but it still feels a bit like QA is being punished for the scheduling issues stemming from F18 | 15:15 |
| jreznik | tflink: this time, not only QA - you can see it on proposed changes | 15:16 |
| jreznik | but back to the question | 15:16 |
| tflink | in this timeframe, I'm skeptical that any formal system for testing will happen | 15:16 |
| tflink | will/could | 15:16 |
| jreznik | we usually don't have any feedback/validation of features/changes that are marked as done, so sometimes it just does not work, fo self contained changes, not a big deal... but once it's in release notes, it should work... if it's in announcement and picked up by medias, we should be sure it works as mentined in proposal etc. | 15:18 |
| tflink | jreznik: sure, I don't think that anyone's arguing with you on that part | 15:18 |
| jreznik | currently, there's no way, once owner says done, we trust them... so at least for system wide, visible changes, we should have at least some smoke testing "it's there and it works" | 15:18 |
| tflink | jreznik: how often and starting when? | 15:19 |
| jreznik | tflink: now the question is, if we have people to do it, if anyone is even willing to do so - as it is in the worst time for qa - during release | 15:19 |
| adamw | what we used to do for features, in a kind of semi-organized way, was look through the list and pick out ones that looked important and ask the feature owners if they wanted to run test days on them | 15:20 |
| adamw | you can kinda see that in the test day list for each release if you look | 15:20 |
| adamw | it's quite a long way from there to some kind of 'validation process' | 15:20 |
| kparal | some changes might be hard to verify, like some SSSD and similar changes - I wonder whether anyone really understands it | 15:21 |
| jreznik | adamw: yeah, and it's great it's covered at least by test days | 15:21 |
| adamw | yeah, that's a good point from kparal | 15:21 |
| adamw | with the best will in the world we don't have expertise on every tech in fedora | 15:22 |
| adamw | i think at least my inclination would be that we're unlikely to be able to commit to some kind of formal testing of all systemwide Changes for f20 whatsoever they may be | 15:22 |
| jreznik | adamw: then maybe test days have even more sense to do it this way and based on results we can say - it's ok/it's not ok | 15:22 |
| adamw | we might be able to try and come up with something more than a test day to check on any really important ones | 15:22 |
| adamw | jreznik: that would be an obvious improvement on the old way, yeah - try and hook the Change process into the results of any test days we run | 15:23 |
| * jsmith likes that idea | 15:23 | |
| jreznik | and I'd be really ok with coverage of the really important ones (big system wide change, change that's going to be featured in the announcement) + for some other test days coverage | 15:24 |
| adamw | OK | 15:24 |
| adamw | so, how about this: once the list of Changes is approved by FESCo we can revisit this topic at a future meeting, with jreznik/fesco present, and come up with a specific plan for what we can try and do for f20 'change validation | 15:25 |
| adamw | we can review how that goes post-f20 | 15:25 |
| tflink | with an emphasis on minimal coverage, I'd say - making it clear that we aren't planning to cover everything | 15:26 |
| jreznik | adamw: I'd be more than happy with it this way for F20 | 15:26 |
| adamw | tflink: agreed | 15:27 |
| adamw | everyone else on board with that? | 15:27 |
| brunowolff | Some changes are more coordination than use observable changes, like the boost change. | 15:27 |
| brunowolff | s/use/user/ | 15:28 |
| kparal | adamw: yes | 15:28 |
| brunowolff | Yeah, start small and see how things go, seems like a good approach. | 15:29 |
| adamw | OK | 15:29 |
| adamw | #agreed we cannot commit to do any form of 'validation testing' for any and all systemwide Changes sight unseen: when FESCo approves the F20 ChangeSet we will re-consider this issue and come up with a realistic plan for testing what we can of the approved Changes | 15:30 |
| adamw | #topic Test Days | 15:30 |
| jreznik | thanks! | 15:30 |
| adamw | mkrizek: ahoy | 15:30 |
| adamw | so now it's been pointed out that the f20 schedule is very tight we probably want to send out a call for test days pretty soon | 15:31 |
| * nirik notes most of the f20 sechedule is still draft, the proposed stuff there is 'no earlier than' | 15:31 | |
| adamw | ah, OK | 15:31 |
| * nirik needs more coffee after that typo. | 15:31 | |
| tflink | but what are the odds that it'll get figured out on wednesday? | 15:32 |
| adamw | still, that was just a side note | 15:32 |
| tflink | assuming that's still when the fesco meetings are | 15:32 |
| adamw | what this topic is really for is https://fedorahosted.org/fedora-qa/ticket/389 | 15:33 |
| brunowolff | I suspect we want to get back to the previous schedule (October and May), so that trying to get this one done in a short time is desireable. | 15:33 |
| tflink | yeah, it seems prudent to start organizing test days sooner than later | 15:33 |
| tflink | adamw: ok, we're getting off topic then :) | 15:33 |
| adamw | we were asked by dpal if we can have an 'out of cycle' test day for some planned FreeIPA enhancements to be added to F19 as updates | 15:33 |
| adamw | #info we were asked by dpal if we can have an 'out of cycle' test day for some planned FreeIPA enhancements to be added to F19 as updates | 15:33 |
| tflink | freeipa strikes me as something which would be difficult to find testers for | 15:34 |
| adamw | i don't see a big problem with that, we can run it just as usual with announcements etc and add it on to the F19 schedule page for future tracking | 15:34 |
| adamw | we've done freeipa test days before | 15:34 |
| tflink | especially AD integration | 15:34 |
| adamw | they seem to like the setup or they wouldn't bother asking for another | 15:34 |
| adamw | i think even if no 'external' testers show up they like to have an organized event for 'internal' testing | 15:34 |
| * tflink doesn't see any problems with the idea of an out-of-cycle test day | 15:34 | |
| adamw | anyone else see any issues with an out-of-cycle test day or should we go ahead and say 'sure'? | 15:35 |
| mkrizek | Martix: test days ^^^ | 15:36 |
| kparal | I don't see a problem | 15:37 |
| tflink | +1 to 'sure' | 15:38 |
| adamw | goddamnit, too many people starting with M. | 15:38 |
| adamw | meetings too early. | 15:38 |
| adamw | adam too idiotic. | 15:38 |
| adamw | well, let's call that good. | 15:39 |
| adamw | #agreed no problems with an out of cycle test day, we'll run it as always and append it to the F19 test day schedule | 15:40 |
| adamw | #info Fedora 20 schedule may be quite tight so we should send out call for F20 test days soon | 15:40 |
| adamw | #topic open floor | 15:40 |
| adamw | so, any other business? | 15:40 |
| brunowolff | This is more for nirik, when are we going to switch over the nightly composes to f20? | 15:41 |
| nirik | already done. ;) | 15:41 |
| nirik | yesterday. | 15:41 |
| nirik | some of them failed compose. | 15:42 |
| brunowolff | Thanks, I'll see if my changes broke anything. | 15:42 |
| nirik | I have one other thing... | 15:42 |
| brunowolff | I also saw that we might be doing weekly install composes. Is that going to start soon? | 15:42 |
| nirik | There's a bunch of fedup bugs coming in. wwoods is not opposed to the idea of people helping triage. If you would like to do so, ping me over on qa or devel and we can try and go thru them. | 15:43 |
| adamw | #info wwoods looking for help triaging fedup bugs, contact nirik to sign up | 15:46 |
| adamw | nirik: any info on the weekly install composes? | 15:48 |
| nirik | dunno. | 15:48 |
| nirik | I think we should really ask anaconda folks what they would like. | 15:49 |
| Martix | mkrizek: hi | 15:49 |
| nirik | daily would actually be easier. | 15:49 |
| nirik | but if they want weekly we could do that I guess. | 15:49 |
| Martix | adamw: out of cycle test day is ok | 15:49 |
| * nirik doesn't know specifically who to ask over there, but someone should. ;) | 15:50 | |
| adamw | Martix: thanks | 15:50 |
| adamw | nirik: bcl is my goto anaconda folk | 15:50 |
| adamw | i'm sure he hates me for that | 15:50 |
| nirik | cool. You want to ask him? or have him drop by fedora-releng and we can hash it out? | 15:51 |
| adamw | well i'd feel odd asking him since i'm not in any way involved in or capable of doing weekly builds... | 15:52 |
| nirik | sure, or I can. | 15:52 |
| adamw | seems a bit unnecessarily go-between-ish | 15:52 |
| nirik | no worries. | 15:52 |
| brunowolff | There is a trac ticket for the install compose stuff. I'll see if I can find it. | 15:52 |
| adamw | right, that'd seem to make more sense. he only bites as much as your AVERAGE anaconda dev. | 15:52 |
| adamw | (wear gloves.) | 15:52 |
| * nirik is happy to talk to most anyone, just thought you might already be talking for some other reason or something... no worries. | 15:53 | |
| brunowolff | https://fedorahosted.org/rel-eng/ticket/5653 | 15:53 |
| adamw | #info nirik will talk to anaconda team about doing regular installer composes (again?) | 15:53 |
| brunowolff | I was insterested in installer composes because I made some changes to the installer ks file to make things better. In particular, you shouldn't have to add a bunch of excludes when overriding repos. | 15:55 |
| brunowolff | I haven't tested that on RHEL 6 though, and this needs a not too old version of pykickstart to work. | 15:56 |
| brunowolff | I was also figuring that this time around, anaconda will probably be a lot more testable throughout the development period. | 15:57 |
| adamw | brunowolff: that'd need a DVD compose, wouldn't it? | 15:57 |
| adamw | the kickstart isn't relevant to netinst? | 15:57 |
| brunowolff | That is correct. | 15:57 |
| adamw | okay. | 15:58 |
| adamw | well, we can try and figure that out. | 15:58 |
| * adamw sets the Quantum Fuse | 15:58 | |
| adamw | any other extremely important business before we all go drink? | 15:59 |
| adamw | alrighty, thanks for coming folks | 16:00 |
| adamw | #endmeeting | 16:00 |
Generated by irclog2html.py 2.12.1 by Marius Gedminas - find it at mg.pov.lt!