From Fedora Project Wiki
Attendees
- adamw (117)
- kparal (37)
- j_dulaney (28)
- tflink (23)
- wwoods (11)
- jskladan (7)
- pschindl (5)
- zodbot (4)
- brunowolff (2)
- nirik (1)
Agenda
Previous meeting follow-up
- on the question of whether we need code hooks in the anaconda UI rewrite to aid automated testing: kparal mailed anaconda-devel-list and wwoods / jlaska directly, but response so far seems thin
- various ideas discussed for moving the topic forward, kparal will continue to try
- j_dulaney is still happy with the test day co-ordination role, adamw is available for help
- adamw has nearly finished the retrospective wrap-up, several tickets filed so far
Revised F17 schedule confirmation
- everyone was happy with the revised schedule proposed by adamw
F16 retrospective activity
- this was shelved until next week as adamw has not finished preparing it and the meeting was running long
AutoQA update
- team is looking into some issues found in testing of the new autoqa version
- most infrastructure has now been upgraded from F14
- resultsdb to begin testing in staging after autoqa 0.7 is deployed
Open floor
Action items
- kparal to continue to look at whether we need hooks in anaconda UI code for testing, will see if anyone in RH's desktop QA team has experience
- adamw to send the (revised schedule) proposal along to rbergeron
IRC Log
adamw | #startmeeting Fedora QA Meeting | 16:00 |
---|---|---|
zodbot | Meeting started Mon Nov 28 16:00:35 2011 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
zodbot | Useful Commands: #action #agreed #halp #info #idea #link #topic. | 16:00 |
adamw | #meetingname fedora-qa | 16:00 |
zodbot | The meeting name has been set to 'fedora-qa' | 16:00 |
adamw | #chair tflink j_dulaney | 16:00 |
zodbot | Current chairs: adamw j_dulaney tflink | 16:00 |
adamw | #topic roll call | 16:00 |
* nirik is lurking around if he can help with anything. | 16:01 | |
* j_dulaney is ready to rock-and-roll! | 16:01 | |
adamw | good morning everybody, welcome to another stonking week of fedora qa | 16:01 |
* pschindl is here | 16:01 | |
* jskladan enters with evil laughter | 16:01 | |
j_dulaney | Uh oh... | 16:02 |
adamw | hey hey hey, that's my job | 16:02 |
* tflink is here | 16:03 | |
adamw | #topic muahahahahahahaahahahah! | 16:03 |
adamw | ahem | 16:03 |
* brunowolff is here | 16:03 | |
adamw | #topic previous meeting follow-up | 16:03 |
adamw | hellooo, everybody. okay, let's see what we had hanging around from last week | 16:03 |
adamw | "kparal to try and co-ordinate all involved in writing installer autoqa tests together to determine if we need any hooks in the new anaconda code" | 16:04 |
adamw | well, it looks like kparal isn't around | 16:04 |
adamw | does anyone know his status on that? | 16:04 |
j_dulaney | #info I think adamw fired kparal last week... | 16:04 |
jskladan | I know that kparal sent out some email to the anaconda guys | 16:04 |
adamw | heh | 16:04 |
jskladan | but I did not see any reply, apart of the ... | 16:05 |
adamw | jskladan: thanks | 16:05 |
* jskladan is looking for it | 16:05 | |
adamw | aha, here he is | 16:05 |
* kparal joins in last | 16:05 | |
kparal | at last | 16:05 |
adamw | kparal: just in time for your starring role! | 16:05 |
adamw | we're on your action item from last week: "kparal to try and co-ordinate all involved in writing installer autoqa tests together to determine if we need any hooks in the new anaconda code" | 16:05 |
kparal | oh yes | 16:05 |
adamw | where'd you get with that? | 16:06 |
kparal | I sent an email to anaconda-devel | 16:06 |
* kparal looking for link | 16:06 | |
pschindl | jskladan is out for a moment (gnome-shell is fell) | 16:06 |
adamw | what? gnome shell crashed?! never! | 16:06 |
pschindl | *fallen :) | 16:06 |
j_dulaney | Not Gnome Shell! | 16:07 |
kparal | #link https://www.redhat.com/archives/anaconda-devel-list/2011-November/msg00112.html | 16:07 |
pschindl | yep, today for second time (propably some poblem with weather applet) | 16:07 |
kparal | pschindl: there are no applets, don't you know? | 16:07 |
kparal | anyway, I started the discussion and a few people responded | 16:07 |
pschindl | sry. extension | 16:07 |
kparal | but it doesn't seem like a huge discussion | 16:08 |
kparal | I hoped for some knowledgeable people to respond, like wwoods or jlaska, because they had experience with the previous test | 16:08 |
adamw | okay | 16:08 |
* j_dulaney only sees clumens | 16:08 | |
adamw | if you're worried the right people aren't replying, probably best poke them directly | 16:08 |
kparal | anyway at least we know the current rats-install test doesn't work :) | 16:08 |
adamw | the thread may have just skipped under their radar | 16:08 |
kparal | I CCed them directly I believe | 16:09 |
adamw | ah okay | 16:09 |
adamw | #info kparal mailed anaconda-devel-list and wwoods / jlaska directly, but response so far seems thin | 16:09 |
j_dulaney | Should we get Máirín Duffy in on this? | 16:09 |
kparal | they know we can respond if they have some questions regarding autoqa | 16:10 |
adamw | not sure mo would have much input? | 16:10 |
j_dulaney | Since she's doing a lot of the design work | 16:10 |
kparal | and that's probably the most important | 16:10 |
adamw | j_dulaney: yeah, but that's at a very high level | 16:10 |
j_dulaney | True | 16:10 |
tflink | I think she's on anaconda-devel but I'm not sure that testable hooks into anaconda is so much of a design level | 16:10 |
kparal | nope | 16:10 |
adamw | j_dulaney: essentially mo draws workflow diagrams, it's still the anaconda team that writes the actual code :) (no insult intended either side) | 16:10 |
* j_dulaney understands now | 16:11 | |
adamw | kparal: well, having a line of communication open here is great, but this doesn't sound like something anaconda team will be actively asking us about | 16:11 |
adamw | it's more a 'we'll put in anything you need but we need you to tell us what it is' deal | 16:11 |
tflink | yeah | 16:11 |
adamw | i'd feel more comfortable if we did at least have a decent idea whether we need stuff in there or not | 16:11 |
kparal | I don't know and finding out is quite an extensive task | 16:12 |
tflink | I'm still not clear on what it is that we would want to do | 16:12 |
adamw | hrm. | 16:12 |
kparal | I hoped just to start the discussion | 16:12 |
adamw | yeah | 16:12 |
tflink | but I'm not sure my questions about purpose are as appropriate for here | 16:13 |
adamw | i guess all we can really do if we don't have any clear ideas is re-ping it | 16:13 |
* jskladan aaaaand I'm back | 16:13 | |
adamw | let's try putting it this way: is the uncertainty here most likely to be a symptom of us *not* needing any special code in anaconda for (auto)qa purposes? | 16:14 |
tflink | no, I don't think so | 16:14 |
tflink | at least not for me | 16:14 |
adamw | well, shucks. :) | 16:14 |
j_dulaney | I'm thinking it would make our life easier | 16:15 |
tflink | my question is "what are we going to try to automate?" | 16:15 |
* j_dulaney is just not sure *what* code is needed, not that it *is* needed | 16:15 | |
kparal | ideally anaconda guys will write their own test suite and hook it up in autoqa. so they should know best | 16:15 |
j_dulaney | kparal: Do they know that they need to do this? | 16:15 |
adamw | did you ask them specifically if they're planning to do that? | 16:15 |
tflink | it sounds like they aren't really sure what would be helpful | 16:15 |
kparal | they already have some test suite in autoqa | 16:16 |
tflink | at least from clumens' response | 16:16 |
j_dulaney | kparal: Won't the new Anaconda render that obsolete? | 16:18 |
kparal | j_dulaney: clumens responded something like they will need to update it | 16:18 |
tflink | j_dulaney: I don't think so. AFAIK, there is no GUI interface in those tests | 16:18 |
kparal | once they start to receive emails with test results they will be more active about it I hope | 16:18 |
adamw | so i'm still kinda looking for a clear way forward here: is the best we can do wait and hope? or re-ping specific people and hope they reply this time? | 16:19 |
kparal | oh, and I don't think we are talking about GUI testing, right? we are talking about anaconda testing | 16:19 |
j_dulaney | How much besides GUI/Workflow is changing? | 16:19 |
tflink | kparal: I thought we were talking about GUI testing since that's the major change for F17, I think | 16:19 |
j_dulaney | How much will that change the backend stuff? | 16:19 |
adamw | i think a lot of it is looking like re-organization rather than change exactly | 16:20 |
adamw | they're moving to a hub / spoke design rather than a serial one | 16:21 |
adamw | (aiui) | 16:21 |
kparal | there might be some misunderstandings here. I believe the new GUI is best tested by humans, and autoqa can be used to test the plumbing | 16:22 |
kparal | if you think we should automate GUI testing, well, I don't think we have the manpower to do that ATM | 16:22 |
adamw | no, that's not it | 16:22 |
tflink | I think we might need to add some clarification to the discussion, then | 16:22 |
adamw | yeah. | 16:23 |
tflink | I'm reading clumens' responses as talking about automated GUI testing | 16:23 |
adamw | gimme a sec to back up and check the history | 16:23 |
adamw | oh, kay | 16:23 |
adamw | sorry, my mistake: we are on UI stuff here | 16:23 |
kparal | as I wrote to clumens, the thing they can do is to allow to control the whole workflow programatically, without clicking buttons. then it can be tested | 16:23 |
adamw | but the thing is that even though we can't automate UI testing *now*, if we need fundamental code in there to make it possible in future, we should ask now | 16:24 |
adamw | because they aren't going to retrofit it. | 16:24 |
tflink | kparal: that's still testing the GUI even if it isn't through screenshots and clicking | 16:24 |
adamw | so yeah, we are specifically on UI here, and i guess part of the problem is that we don't have any UI tests so it's hard to know what we'd need | 16:24 |
kparal | that means we have to have someone who understands UI testing | 16:25 |
tflink | specifically, automated and scripted UI testing | 16:25 |
adamw | yeah. i think what we're missing here is the experience of trying to write automated UI tests against an existing codebase and going 'damn, wouldn't it be great if i could XXX' | 16:25 |
j_dulaney | Indeed | 16:26 |
adamw | kparal: tflink: do you know of anyone who fits the bill? | 16:26 |
kparal | I can talk to the RH desktop team and ask | 16:27 |
adamw | yeah, that might be a good way to go. | 16:27 |
kparal | we have a few guys sitting here in Brno | 16:27 |
adamw | sorry to drag this item out so far, but it'd be nice to have a definite plan. | 16:27 |
tflink | adamw: no not really, I've heard about building in testable interfaces but have never dealt with them myself and I don't know of anyone who has | 16:27 |
j_dulaney | Would anyone outside of Fedora have this experience? | 16:28 |
* wwoods reading scrollback | 16:29 | |
adamw | j_dulaney: well, probably? but we'd need someone who'd talk to us and be able to explain what we'd need | 16:29 |
adamw | oh hey, wwoods | 16:29 |
adamw | wwoods: executive summary: clumens has told us that if we want hooks for automated testing in the new UI code we should ask for them now so they get baked in at the start, but we're really not sure what would be useful as we don't really have any automated UI testing experience | 16:29 |
adamw | wwoods: kparal was hoping you might have a few thoughts | 16:30 |
wwoods | generally UI automation seems to use a11y | 16:31 |
adamw | (accessibility, for those playing along at home) | 16:32 |
wwoods | right. I can't remember what the current libraries are for this stuff | 16:32 |
wwoods | but from the anaconda side - we just have to make sure we write a proper accessibility-enabled UI | 16:32 |
wwoods | with the UI rewrite we're taking every step to use the standard widgets where possible | 16:33 |
adamw | okay | 16:33 |
wwoods | and we're trying to make sure the UI can be run on a normal system in a nice test mode | 16:33 |
wwoods | so it should be pretty easy to mess around and see how/if you can automate it | 16:33 |
wwoods | I think we can commit to making sure the UI is a11y-enabled (whatever that might entail) | 16:33 |
adamw | okay | 16:34 |
adamw | i think we're not necessarily going to get much farther in here | 16:34 |
wwoods | but I'll definitely bring it up with the team and we'll talk further on the mailing list(s) and/or put something in the UI rewrite docs | 16:34 |
* j_dulaney thinks that it comes down to what tests we want to run | 16:34 | |
adamw | kparal: okay if i action this back to you to try and follow up this week? | 16:34 |
kparal | yes | 16:35 |
adamw | okay | 16:35 |
wwoods | basically what it comes down to is: nearly automated GUI testing uses the a11y stuff, so if you want to be able to automate the UI itself, we need to have a11y support | 16:35 |
kparal | wwoods: just please respond with your thoughts to my email in anaconda-devel, thanks | 16:36 |
adamw | #action kparal to continue to look at whether we need hooks in anaconda UI code for testing, will see if anyone in RH's desktop QA team has experience | 16:36 |
wwoods | kparal: will do. | 16:36 |
adamw | thanks wwoods | 16:36 |
adamw | okay, let's try and blow through the rest of this fast :) | 16:36 |
adamw | "j_dulaney to take a shot at co-ordinating F17 test days, thanks! " | 16:36 |
adamw | j_dulaney: I sent you a sort of 'getting started' guide, did you get that? | 16:37 |
j_dulaney | I did | 16:37 |
adamw | cool | 16:37 |
* j_dulaney has been on vacation | 16:37 | |
j_dulaney | I'll get to it this afternoon | 16:37 |
adamw | that's fine | 16:37 |
adamw | just wanted to make sure communications are functioning properly etc :) | 16:37 |
adamw | alright, so nothing further necessary there | 16:38 |
adamw | #info j_dulaney is still happy with the test day co-ordination role, adamw is available for help | 16:38 |
adamw | "adamw to get retrospective tickets filed and summary mailed this week" | 16:38 |
adamw | i'm not quite done with this, but close | 16:38 |
adamw | i'm not gonna go into huge detail because it'd just duplicate the summary mail, but i've been filing tickets on our trac and releng's, and sending out a few ml discussions | 16:39 |
adamw | most of the tickets are just up for grabs, so do feel free to grab any you feel like taking a shot at | 16:39 |
adamw | QA tickets so far are 256 through 260 | 16:40 |
adamw | #info adamw has nearly finished the retrospective wrap-up, several tickets filed so far | 16:40 |
adamw | anyone have any concerns / questions there or should we roll along? | 16:41 |
j_dulaney | Rolling on the river | 16:42 |
adamw | rolling rolling rolling | 16:42 |
adamw | rawhiiiiiiiiide! | 16:42 |
adamw | "adamw to start an ml thread proposing changes to the tc dates for f17 schedule to be sent to rbergeron " | 16:42 |
adamw | we've got an agenda item for that later, so let's table it | 16:42 |
adamw | aaand here it comes! | 16:42 |
adamw | #topic Revised F17 schedule confirmation | 16:43 |
j_dulaney | Ponies! | 16:43 |
adamw | so i did indeed start a thread proposing revisions to the TC drop dates for f17 | 16:43 |
adamw | https://lists.fedoraproject.org/pipermail/test/2011-November/104583.html | 16:43 |
adamw | it looks like everyone was happy with that, so i'm planning to ask the berge to put it in place - just wanted to check if there were any last-minute issues anyone saw | 16:43 |
j_dulaney | Not here | 16:44 |
adamw | speak now or forever hold thy peace | 16:44 |
adamw | okay! | 16:45 |
adamw | #agreed proposed revisions to f17 schedule look good | 16:45 |
adamw | #action adamw to send the proposal along to rbergeron | 16:45 |
adamw | #topic F16 retrospective activity | 16:45 |
adamw | oh, yeah, i did want to take a few minutes just to flag up the QA retrospective tickets i filed so far so everyone knows the kinds of stuff we need to take care of ahead of f17 alpha cycle starting up | 16:46 |
adamw | it might be best to shelve this for next week though as there's some stuff that might start up interesting discussions but we're kinda long already | 16:47 |
adamw | is that okay? or was anyone really hoping for a nice long meeting? :) | 16:47 |
tflink | works for me | 16:48 |
* j_dulaney is good | 16:48 | |
adamw | #agreed this topic shelved till next week since not all tickets are yet filed and this meeting is running long | 16:48 |
adamw | #topic AutoQA update | 16:48 |
adamw | anything exciting from the world of autoqa, folks? | 16:48 |
tflink | still looking to get the new version of autoqa deployed | 16:49 |
kparal | we are just in the process of reviewing staging results from the past week and the weekend | 16:49 |
kparal | we found some weird issues | 16:49 |
kparal | it seems mainly autotest related, but are not sure | 16:50 |
kparal | *we | 16:50 |
kparal | I updated documentation about autoqa configuration | 16:50 |
adamw | #info team is looking into some issues found in testing of the new autoqa version | 16:50 |
kparal | if we don't find anything critical I would like to deploy _soon_ | 16:51 |
kparal | who's got more updates? | 16:51 |
jskladan | pschindl & I are once again working on the ResultsDB (querying api and the frontend) | 16:51 |
tflink | most of the production infrastructure has been updated to not run on F14 any more | 16:52 |
jskladan | and IIRC, after we deploy the new version, we'll start testing ResultsDB on our Staging setup | 16:52 |
adamw | #info most infrastructure has now been upgraded from F14 | 16:52 |
adamw | #info resultsdb to begin testing in staging after autoqa 0.7 is deployed | 16:52 |
adamw | sounds good | 16:53 |
adamw | okay, that's all from autoqa? | 16:54 |
tflink | I think so | 16:54 |
adamw | cool | 16:54 |
adamw | thanks | 16:54 |
adamw | #topic open floor | 16:54 |
adamw | aaand we made it - any other stuff? | 16:54 |
* adamw notes that you can always add agenda topics to the wiki page ahead of time, which will likely give them more time than the 'open floor' slot | 16:55 | |
adamw | PSA ends! | 16:55 |
j_dulaney | No one was fired? | 16:55 |
j_dulaney | PSA? | 16:55 |
tflink | public service announcement | 16:56 |
adamw | j_dulaney: i can fire you if you think it would help. | 16:56 |
j_dulaney | I've already been fired | 16:56 |
adamw | hey, that doesn't stop me. ask tflink, he must've been fired dozens of times now. | 16:58 |
* tflink has lost count | 16:58 | |
brunowolff | FYI rawhide is a bit rough now. gnome-shell has a soname issue, so it will be hard to test fresh builds. Hopefully this gets fixed soon. | 16:58 |
adamw | usually does. | 16:59 |
adamw | okay, sounds like we're about done here | 16:59 |
adamw | thanks for coming, everyone | 16:59 |
adamw | #endmeeting | 16:59 |
Generated by irclog2html.py 2.8 by Marius Gedminas - find it at mg.pov.lt!