From Fedora Project Wiki

< QA‎ | Meetings

Attendees

  • adamw (77)
  • nirik (21)
  • tflink (20)
  • jreznik (18)
  • brunowolff (16)
  • kparal (9)
  • zodbot (4)
  • jsmith (3)
  • Martix (2)
  • mkrizek (2)
  • Cerlyn (1)
  • pschindl (1)

Agenda

  • Fedora 20 Change validation
  • Test Days
  • Open floor

Fedora 20 Change validation

  • At last week's meeting jreznik requested validation of at least systemwide Changes for F20: can we do this?
  • After some discussion, it was agreed that we cannot commit to do any form of 'validation testing' for any and all systemwide Changes sight unseen: when FESCo approves the F20 ChangeSet we will re-consider this issue and come up with a realistic plan for testing what we can of the approved Changes

Test Days

  • We were asked by dpal if we can have an 'out of cycle' test day for some planned FreeIPA enhancements to be added to F19 as updates
  • We agreed there are no problems with an out of cycle test day, we'll run it as always and append it to the F19 test day schedule
  • Fedora 20 schedule may be quite tight so we should send out call for F20 test days soon

Open floor

  • wwoods was looking for help triaging fedup bugs, contact nirik to sign up
  • nirik will talk to anaconda team about doing regular installer composes (again?)

Action items

N/A

IRC Log

adamw #startmeeting Fedora QA meeting 15:04
zodbot Meeting started Mon Jul 8 15:04:26 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:04
zodbot Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:04
adamw #meetingname fedora-qa 15:04
zodbot The meeting name has been set to 'fedora-qa' 15:04
adamw #topic Roll call 15:04
* kparal here 15:04
* mkrizek is here 15:04
* Cerlyn is here 15:04
adamw sorry for the late start, folks - who's around for funtimes? 15:04
* tflink is here 15:05
* nirik is lurking. 15:05
* brunowolff is here 15:05
adamw yay funtimes 15:06
* jsmith is lurking 15:07
adamw like a ninja 15:07
jsmith I'm sneaky like that :-) 15:07
adamw alrighty! 15:07
adamw well, all i had for the agenda was a couple of things left from last time 15:08
adamw #topic Fedora 20 Change validation 15:08
adamw #info last week, jreznik asked if it was possible for us to do some form of 'validation testing' for the systemwide Changes for Fedora 20 15:08
kparal which changes? 15:09
adamw i figured we'd better discuss it 15:09
adamw kparal: Changes as in 'what used to be called Features' 15:09
kparal ah 15:09
adamw one of the problems is that AFAIK we don't have a list of them all yet 15:09
kparal validation after they are deemed complete, or before? 15:09
nirik the deadline is the 16th. 15:10
tflink and "some form of validation testing" is very vague 15:10
adamw well, that's one of the things to discuss, i guess. 15:10
tflink nirik: the 16th of july? 15:11
* jreznik is here 15:11
adamw jreznik: is https://fedoraproject.org/wiki/Category:ChangeReadyForWrangler really all the Changes that are proposed right now? 15:11
nirik yes. 15:11
nirik https://fedoraproject.org/wiki/Releases/20/Schedule 15:11
* pschindl is here. 15:11
* adamw notes https://fedoraproject.org/wiki/Category:SystemWideChanges is empty 15:11
nirik so, they arent even all in yet. 15:11
adamw right, which makes it hard to eyeball a list and come up with an answer 15:11
jreznik https://fedoraproject.org/wiki/Category:ChangeAcceptedF20 for accepted 15:11
adamw jreznik: did you have a more specific idea in mind for exactly what you'd like us to be doing here? 15:12
* adamw brb, call of nature 15:12
tflink wow, I didn't realize that the downtime between f19 and f20 was so short 15:12
adamw #chair kparal brunowolff 15:12
zodbot Current chairs: adamw brunowolff kparal 15:12
jreznik https://fedoraproject.org/wiki/Category:SystemWideChange should not be empty (it's not...) 15:12
adamw oh, I missed the 's' 15:13
adamw er, added an s. 15:13
jreznik tflink: it's really short, that's probably the reason why not so many proposals were submitted so far 15:13
jreznik and also f18, f19 were big, f20 seems to be smaller 15:13
kparal I assumed we're branching in September? 15:13
brunowolff August 8th I think. 15:13
tflink kparal: nope, august 06 15:13
* adamw notes that https://fedoraproject.org/wiki/Releases/20/ChangeSet has the incorrect category names 15:13
brunowolff Alpha freeze is Augist 20th I think. 15:13
kparal wow, the world is speeding up 15:13
* tflink should realy start paying more attention to fesco, this is going to impact automation plans 15:14
jreznik adamw: for ChangeSet - should be ready this week, I need to tweak my old scripts :( 15:14
jreznik tflink: the main reason why fesco asked me for this schedule was to avoid christmas holidays (there's approx 1 month buffer now) 15:15
jreznik and it's not final schedule - I think FESCo would listen to QA if you would need more time (and I'd say - propose QA automation as Change...) 15:15
tflink jreznik: yeah, I understand that but it still feels a bit like QA is being punished for the scheduling issues stemming from F18 15:15
jreznik tflink: this time, not only QA - you can see it on proposed changes 15:16
jreznik but back to the question 15:16
tflink in this timeframe, I'm skeptical that any formal system for testing will happen 15:16
tflink will/could 15:16
jreznik we usually don't have any feedback/validation of features/changes that are marked as done, so sometimes it just does not work, fo self contained changes, not a big deal... but once it's in release notes, it should work... if it's in announcement and picked up by medias, we should be sure it works as mentined in proposal etc. 15:18
tflink jreznik: sure, I don't think that anyone's arguing with you on that part 15:18
jreznik currently, there's no way, once owner says done, we trust them... so at least for system wide, visible changes, we should have at least some smoke testing "it's there and it works" 15:18
tflink jreznik: how often and starting when? 15:19
jreznik tflink: now the question is, if we have people to do it, if anyone is even willing to do so - as it is in the worst time for qa - during release 15:19
adamw what we used to do for features, in a kind of semi-organized way, was look through the list and pick out ones that looked important and ask the feature owners if they wanted to run test days on them 15:20
adamw you can kinda see that in the test day list for each release if you look 15:20
adamw it's quite a long way from there to some kind of 'validation process' 15:20
kparal some changes might be hard to verify, like some SSSD and similar changes - I wonder whether anyone really understands it 15:21
jreznik adamw: yeah, and it's great it's covered at least by test days 15:21
adamw yeah, that's a good point from kparal 15:21
adamw with the best will in the world we don't have expertise on every tech in fedora 15:22
adamw i think at least my inclination would be that we're unlikely to be able to commit to some kind of formal testing of all systemwide Changes for f20 whatsoever they may be 15:22
jreznik adamw: then maybe test days have even more sense to do it this way and based on results we can say - it's ok/it's not ok 15:22
adamw we might be able to try and come up with something more than a test day to check on any really important ones 15:22
adamw jreznik: that would be an obvious improvement on the old way, yeah - try and hook the Change process into the results of any test days we run 15:23
* jsmith likes that idea 15:23
jreznik and I'd be really ok with coverage of the really important ones (big system wide change, change that's going to be featured in the announcement) + for some other test days coverage 15:24
adamw OK 15:24
adamw so, how about this: once the list of Changes is approved by FESCo we can revisit this topic at a future meeting, with jreznik/fesco present, and come up with a specific plan for what we can try and do for f20 'change validation 15:25
adamw we can review how that goes post-f20 15:25
tflink with an emphasis on minimal coverage, I'd say - making it clear that we aren't planning to cover everything 15:26
jreznik adamw: I'd be more than happy with it this way for F20 15:26
adamw tflink: agreed 15:27
adamw everyone else on board with that? 15:27
brunowolff Some changes are more coordination than use observable changes, like the boost change. 15:27
brunowolff s/use/user/ 15:28
kparal adamw: yes 15:28
brunowolff Yeah, start small and see how things go, seems like a good approach. 15:29
adamw OK 15:29
adamw #agreed we cannot commit to do any form of 'validation testing' for any and all systemwide Changes sight unseen: when FESCo approves the F20 ChangeSet we will re-consider this issue and come up with a realistic plan for testing what we can of the approved Changes 15:30
adamw #topic Test Days 15:30
jreznik thanks! 15:30
adamw mkrizek: ahoy 15:30
adamw so now it's been pointed out that the f20 schedule is very tight we probably want to send out a call for test days pretty soon 15:31
* nirik notes most of the f20 sechedule is still draft, the proposed stuff there is 'no earlier than' 15:31
adamw ah, OK 15:31
* nirik needs more coffee after that typo. 15:31
tflink but what are the odds that it'll get figured out on wednesday? 15:32
adamw still, that was just a side note 15:32
tflink assuming that's still when the fesco meetings are 15:32
adamw what this topic is really for is https://fedorahosted.org/fedora-qa/ticket/389 15:33
brunowolff I suspect we want to get back to the previous schedule (October and May), so that trying to get this one done in a short time is desireable. 15:33
tflink yeah, it seems prudent to start organizing test days sooner than later 15:33
tflink adamw: ok, we're getting off topic then :) 15:33
adamw we were asked by dpal if we can have an 'out of cycle' test day for some planned FreeIPA enhancements to be added to F19 as updates 15:33
adamw #info we were asked by dpal if we can have an 'out of cycle' test day for some planned FreeIPA enhancements to be added to F19 as updates 15:33
tflink freeipa strikes me as something which would be difficult to find testers for 15:34
adamw i don't see a big problem with that, we can run it just as usual with announcements etc and add it on to the F19 schedule page for future tracking 15:34
adamw we've done freeipa test days before 15:34
tflink especially AD integration 15:34
adamw they seem to like the setup or they wouldn't bother asking for another 15:34
adamw i think even if no 'external' testers show up they like to have an organized event for 'internal' testing 15:34
* tflink doesn't see any problems with the idea of an out-of-cycle test day 15:34
adamw anyone else see any issues with an out-of-cycle test day or should we go ahead and say 'sure'? 15:35
mkrizek Martix: test days ^^^ 15:36
kparal I don't see a problem 15:37
tflink +1 to 'sure' 15:38
adamw goddamnit, too many people starting with M. 15:38
adamw meetings too early. 15:38
adamw adam too idiotic. 15:38
adamw well, let's call that good. 15:39
adamw #agreed no problems with an out of cycle test day, we'll run it as always and append it to the F19 test day schedule 15:40
adamw #info Fedora 20 schedule may be quite tight so we should send out call for F20 test days soon 15:40
adamw #topic open floor 15:40
adamw so, any other business? 15:40
brunowolff This is more for nirik, when are we going to switch over the nightly composes to f20? 15:41
nirik already done. ;) 15:41
nirik yesterday. 15:41
nirik some of them failed compose. 15:42
brunowolff Thanks, I'll see if my changes broke anything. 15:42
nirik I have one other thing... 15:42
brunowolff I also saw that we might be doing weekly install composes. Is that going to start soon? 15:42
nirik There's a bunch of fedup bugs coming in. wwoods is not opposed to the idea of people helping triage. If you would like to do so, ping me over on qa or devel and we can try and go thru them. 15:43
adamw #info wwoods looking for help triaging fedup bugs, contact nirik to sign up 15:46
adamw nirik: any info on the weekly install composes? 15:48
nirik dunno. 15:48
nirik I think we should really ask anaconda folks what they would like. 15:49
Martix mkrizek: hi 15:49
nirik daily would actually be easier. 15:49
nirik but if they want weekly we could do that I guess. 15:49
Martix adamw: out of cycle test day is ok 15:49
* nirik doesn't know specifically who to ask over there, but someone should. ;) 15:50
adamw Martix: thanks 15:50
adamw nirik: bcl is my goto anaconda folk 15:50
adamw i'm sure he hates me for that 15:50
nirik cool. You want to ask him? or have him drop by fedora-releng and we can hash it out? 15:51
adamw well i'd feel odd asking him since i'm not in any way involved in or capable of doing weekly builds... 15:52
nirik sure, or I can. 15:52
adamw seems a bit unnecessarily go-between-ish 15:52
nirik no worries. 15:52
brunowolff There is a trac ticket for the install compose stuff. I'll see if I can find it. 15:52
adamw right, that'd seem to make more sense. he only bites as much as your AVERAGE anaconda dev. 15:52
adamw (wear gloves.) 15:52
* nirik is happy to talk to most anyone, just thought you might already be talking for some other reason or something... no worries. 15:53
brunowolff https://fedorahosted.org/rel-eng/ticket/5653 15:53
adamw #info nirik will talk to anaconda team about doing regular installer composes (again?) 15:53
brunowolff I was insterested in installer composes because I made some changes to the installer ks file to make things better. In particular, you shouldn't have to add a bunch of excludes when overriding repos. 15:55
brunowolff I haven't tested that on RHEL 6 though, and this needs a not too old version of pykickstart to work. 15:56
brunowolff I was also figuring that this time around, anaconda will probably be a lot more testable throughout the development period. 15:57
adamw brunowolff: that'd need a DVD compose, wouldn't it? 15:57
adamw the kickstart isn't relevant to netinst? 15:57
brunowolff That is correct. 15:57
adamw okay. 15:58
adamw well, we can try and figure that out. 15:58
* adamw sets the Quantum Fuse 15:58
adamw any other extremely important business before we all go drink? 15:59
adamw alrighty, thanks for coming folks 16:00
adamw #endmeeting 16:00

Generated by irclog2html.py 2.12.1 by Marius Gedminas - find it at mg.pov.lt!