From Fedora Project Wiki

< SIGs‎ | bigdata

(Created page with "<mattf> who's here for the big data sig meeting? * rsquared raises his hand * orc_emac_ is now known as orc_fedo * opuk (~kupo@h-1-5.a176.priv.bahnhof.se) has joined #fedora-m...")
 
No edit summary
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
<mattf> who's here for the big data sig meeting?
<nowiki> <mattf> morning tim </nowiki>
* rsquared raises his hand
 
* orc_emac_ is now known as orc_fedo
<nowiki> masta mattf misc mjg59 mkolman mkolman_ muep </nowiki>
* opuk (~kupo@h-1-5.a176.priv.bahnhof.se) has joined #fedora-meeting-1
 
* besser82 (~besser82@fedora/besser82) has joined #fedora-meeting-1
<nowiki> <tstclair> morning mattf </nowiki>
* gil__ (~quassel@151.60.14.100) has joined #fedora-meeting-1
 
<gil__> hi
<nowiki> <mattf> who's here for the big data sig meeting? </nowiki>
<tstclair> Morning gil__
 
* mizdebsk (mizdebsk__@nat/redhat/x-qpgqpnidykcdkxrl) has joined #fedora-meeting-1
<nowiki> * rsquared raises his hand </nowiki>
<tstclair> Morning Folks, I figured I would collect a list of agenda items:  1.) Status 2.) Issues / Blockers 3.) Feel free to add to the list.
 
<samkottler> tstclair: do we want to use meetbot?
<nowiki> * orc_emac_ is now known as orc_fedo </nowiki>
<mattf> samkottler, good idea
 
<mattf> first meeting hiccups
<nowiki> * opuk (~kupo@h-1-5.a176.priv.bahnhof.se) has joined #fedora-meeting-1 </nowiki>
<tstclair> samkottler, 1st time doing this..
 
<gil__> tstclair: +1
<nowiki> * besser82 (~besser82@fedora/besser82) has joined #fedora-meeting-1 </nowiki>
<samkottler> tstclair: hehe no worries :)
 
* suehle has quit (Ping timeout: 240 seconds)
<nowiki> * gil__ (~quassel@151.60.14.100) has joined #fedora-meeting-1 </nowiki>
<samkottler> tstclair: http://meetbot.debian.net/Manual.html is a good intro
 
<tstclair> I thought this channel was always recorded according to the docs.  
<nowiki> <gil__> hi </nowiki>
<tstclair> I think we can figure out logistics for next round.
 
<tstclair> So current question: is https://fedoraproject.org/wiki/SIGs/bigdata/packaging up to date?  And are there other items that folks thave been looking into?
<nowiki> <tstclair> Morning gil__ </nowiki>
* witlessb (~wb@cpc29-lewi15-2-0-cust72.2-4.cable.virginm.net) has joined #fedora-meeting-1
 
<tstclair> witlessb, <tstclair> So current question: is https://fedoraproject.org/wiki/SIGs/bigdata/packaging up to date?  And are there other items that folks thave been looking into?
<nowiki> * mizdebsk (mizdebsk__@nat/redhat/x-qpgqpnidykcdkxrl) has joined #fedora-meeting-1 </nowiki>
<mattf> gil__, i added solr to the packaging list when you announced it on the mailing list, fyi
 
<gil__> ok
<nowiki> <tstclair> Morning Folks, I figured I would collect a list of agenda items:  1.) Status 2.) Issues / Blockers 3.) Feel free to add to the list. </nowiki>
<mattf> tstclair, it's up2date wrt what i'm working on
 
<tstclair> To give a brief synopsis:  I've submitted curator and updated tachyon to build against the latest 0.4.0 branch.  There were several issues in testing and I have to patches that hack around the issues and opened tickets upstream .  
<nowiki> <samkottler> tstclair: do we want to use meetbot? </nowiki>
<tstclair> Are there any new developments that are not on the list?
 
<gil__> ... again some BR from finish to review....
<nowiki> <mattf> samkottler, good idea </nowiki>
<gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1002154
 
<tstclair> besser82, witlessb, rsquared, pmackinn  any updates / issues ?
<nowiki> <mattf> first meeting hiccups </nowiki>
<gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1002170
 
<gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1005904
<nowiki> <tstclair> samkottler, 1st time doing this.. </nowiki>
<rsquared> Developments as in new bits being packaged or updates on existing packages?
 
<mizdebsk> i'm trying to find time to work on better Ivy integration, which should improve packaging of some applications (Scala stuff and others)
<nowiki> <gil__> tstclair: +1 </nowiki>
randomuser` rbergeron rdieter robyduck roshi rsquared
 
<tstclair> rsquared, both.
<nowiki> <samkottler> tstclair: hehe no worries :) </nowiki>
<besser82> tstclair: actually no issues / updates from me...
 
<pmackinn> tstclair, nothing here
<nowiki> * suehle has quit (Ping timeout: 240 seconds) </nowiki>
<witlessb> tstclair: none from me
 
<tstclair> mizdebsk, I'm sure pmackinn and willb would be interested there.
<nowiki> <samkottler> tstclair: http://meetbot.debian.net/Manual.html is a good intro </nowiki>
<mattf> tstclair, did zookeeper get settled?
 
<rsquared> hadoop 2.2.0 in F20 has a minor issue with the hadoop-common testing jar/pom. It's misnamed. I've got a fix I can spin if it's blocking anyone.
<nowiki> <tstclair> I thought this channel was always recorded according to the docs. </nowiki>
<tstclair> I took ownership, I'm heavily dependent..  
 
* msrb (msrb@nat/redhat/x-sllcvrytyozxcsgd) has joined #fedora-meeting-1
<nowiki> <tstclair> I think we can figure out logistics for next round. </nowiki>
<tstclair> both tachyon and mesos require.  
 
masta mattf misc mizdebsk mjg59 mkolman mkolman_ msrb muep
<nowiki> <tstclair> So current question: is https://fedoraproject.org/wiki/SIGs/bigdata/packaging up to date?  And are there other items that folks thave been looking into? </nowiki>
<tstclair> mattf, ^
 
<mattf> gotcha
<nowiki> * witlessb (~wb@cpc29-lewi15-2-0-cust72.2-4.cable.virginm.net) has joined #fedora-meeting-1 </nowiki>
<tstclair> are some of our friends from IBM in the audience?
 
<rsquared> masta: FYI, guava update coming in rawhide
<nowiki> <tstclair> witlessb, <tstclair> So current question: is https://fedoraproject.org/wiki/SIGs/bigdata/packaging up to date?  And are there other items that folks thave been looking into? </nowiki>
<mizdebsk> i'll work on guava next week
 
<gil__> lucene-spatial https://bugzilla.redhat.com/show_bug.cgi?id=1002154
<nowiki> <mattf> gil__, i added solr to the packaging list when you announced it on the mailing list, fyi </nowiki>
<tstclair> Second Item:  Are there any issues folks are blocked on or need assistance with. 
 
<tstclair> ?
<nowiki> <gil__> ok </nowiki>
<mattf> i'm not blocked
 
<gil__> lucene-analysis-morfologik https://bugzilla.redhat.com/show_bug.cgi?id=1002170
<nowiki> <mattf> tstclair, it's up2date wrt what i'm working on </nowiki>
<tstclair> I'll follow up with willb tomorrow, but I know there were issues with scala and sbt.
 
<gil__> solr core  https://bugzilla.redhat.com/show_bug.cgi?id=1005904
<nowiki> <tstclair> To give a brief synopsis: I've submitted curator and updated tachyon to build against the latest 0.4.0 branch. There were several issues in testing and I have to patches that hack around the issues and opened tickets upstream . </nowiki>
<pmackinn> tstclair, https://bugzilla.redhat.com/show_bug.cgi?id=1010383 to start, fpc exception request filed
 
<rsquared> hadoop-2.2.0 is currently blocked in rawhide by bz1023116.  mizdebsk knows about it and is looking into it
<nowiki> <tstclair> Are there any new developments that are not on the list? </nowiki>
<mattf> gil__, will you add those to the packages in review or to the review tracker?
 
<gil__> just the 1^
<nowiki> <gil__> ... again some BR from finish to review.... </nowiki>
<gil__> ops
 
<tstclair> I have no blockers.
<nowiki> <gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1002154 </nowiki>
<tstclair> Item 3: Any new developments in the BIG-DATA ecosystem that are worth watching? 
 
<tstclair> I've seen a lot of new ML coming into the works, no clear leaders.
<nowiki> <tstclair> besser82, witlessb, rsquared, pmackinn  any updates / issues ? </nowiki>
<gil__> i have only a problem with solr contib module:
 
<tstclair> A lot of stuff based on Spark 0.8+
<nowiki> <gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1002170 </nowiki>
<rsquared> HBase has a few dependencies awaiting review and I need to submit the jamon packages as well.
 
<rsquared> HBase itself is compiling but need eval of test suite
<nowiki> <gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1005904 </nowiki>
<mattf> at hadoop world this week i had a discussion and the topic of data nucleus came up. turns out it's something the community would like to get rid of.
 
<gil__> clustering require carrot2-mini-3.8.0 (this one depend with non-free mahout-math-0.6)
<nowiki> <rsquared> Developments as in new bits being packaged or updates on existing packages? </nowiki>
<rsquared> pmackinn: I'm sure you're glad to hear that about data nucleus
 
<pmackinn> i will eagerly follow upstream's lead in this regard
<nowiki> <mizdebsk> i'm trying to find time to work on better Ivy integration, which should improve packaging of some applications (Scala stuff and others)</nowiki>
<mattf> ok, i'll give it a few days and see what shakes out
</nowiki>
<tstclair> Is jline2 an issue for anyone, I have not dug in there.
 
<gil__> +1
<nowiki> randomuser` rbergeron rdieter robyduck roshi rsquared </nowiki>
<mizdebsk> jline stuff should be no problem, there is either 1.x or 2.x to choose from
 
<rsquared> I'm not sure how much jline is used in hadoop, but I am expecting jline 2 to cause a problem.  Unsure if I'll need to rely on the compat package
<nowiki> <tstclair> rsquared, both. </nowiki>
<tstclair> mizdebsk, for now... I thought that there was the intention of deprecating.
 
<rsquared> tstclair: There's also netty and guava changes coming
<nowiki> <besser82> tstclair: actually no issues / updates from me... </nowiki>
<rsquared> I expect both to be an issue for hadoop
 
<mizdebsk> ideally, yes, i can't say much, but i've ported 2 packages to jline 2.10 and this was just a simple namespace change
<nowiki> <pmackinn> tstclair, nothing here </nowiki>
<tstclair> do we need compat packages?
 
<gil__> netty breaks some thing also on gradle
<nowiki> <witlessb> tstclair: none from me </nowiki>
<mizdebsk> for guava, i'll investigate
 
<mizdebsk> netty already has a compat package (netty31), netty will be updated to 4.x
<nowiki> <tstclair> mizdebsk, I'm sure pmackinn and willb would be interested there. </nowiki>
<tstclair> rsquared, ^
 
<gil__> for gradle netty31 is unusable
<nowiki> <mattf> tstclair, did zookeeper get settled? </nowiki>
<rsquared> The maintainer of netty said he doesn't plan to maintain a compat package
 
<mizdebsk> rsquared: i am maintainer of netty and there already is a compat package
<nowiki> <rsquared> hadoop 2.2.0 in F20 has a minor issue with the hadoop-common testing jar/pom. It's misnamed. I've got a fix I can spin if it's blocking anyone. </nowiki>
<tstclair> well don't they usually hang on until the deps transition?
 
<gil__> littleproxy is the library which caause the problem with gradle/netty
<nowiki> <tstclair> I took ownership, I'm heavily dependent.. </nowiki>
<rsquared> mizdebsk: who is Jon VanAlten then?  He sent the e-mails about the new netty and said he didn't plan to maintain a compat version
 
<mizdebsk> tstclair: of course, for reasonable time, but i cannot wait for months
<nowiki> * msrb (msrb@nat/redhat/x-sllcvrytyozxcsgd) has joined #fedora-meeting-1 </nowiki>
<mizdebsk> rsquared: he is a comaintainer
 
<rsquared> hadoop uses the current netty (3.6).  Not sure if 3.1 will work. Is there need/plan for a netty 3.6 compat package?
<nowiki> <tstclair> both tachyon and mesos require. </nowiki>
<gil__> newer littleproxy required some libraries wich dont work and dont compile under java7
</nowiki>
<tstclair> mizdebsk, what is a reasonable time?
 
<gil__> https://github.com/barchart/barchart-udt/
<nowiki> masta mattf misc mizdebsk mjg59 mkolman mkolman_ msrb muep </nowiki>
<mizdebsk> tstclair: depends on the case, if i can see there are difficulties and there is some progress, i can wait
 
<gil__> Java wrapper for native C++ UDT protocol require also for build native libraries
<nowiki> <tstclair> mattf, ^ </nowiki>
<mizdebsk> otherwise i'm updating; maintainers of dependant packages can always fork the old version and convert it to compat package
 
<gil__> org.codeswarm:maven-nar-plugin:20121119 which is not compatible with our surefire version
<nowiki> <mattf> gotcha </nowiki>
<rsquared> gil__: Is gradle planning to be dropped?
 
* aeperezt (~aeperezt@fedora/aeperezt) has joined #fedora-meeting-1
<nowiki> <tstclair> are some of our friends from IBM in the audience? </nowiki>
<gil__> yes
 
<gil__> rsquared: there are too many problems with gradle at the moment
<nowiki> <rsquared> masta: FYI, guava update coming in rawhide </nowiki>
<gil__> asm conflict
 
<gil__> newer eclipse aether apis
<nowiki> <mizdebsk> i'll work on guava next week </nowiki>
<rsquared> mizdebsk: What's the typical timeline line for updates like these?  If a package breaks in rawhide because of a dep update is there a problem so long as it's fixed by next release cycle?
 
<gil__> please mizdebsk,remember if i've forgotten something...
<nowiki> <gil__> lucene-spatial https://bugzilla.redhat.com/show_bug.cgi?id=1002154 </nowiki>
<mizdebsk> rsquared: according to the policy, announce 1 week in advance and then you can update
 
<mizdebsk> (that's why i wait with guava for next week)
<nowiki> <tstclair> Second Item:  Are there any issues folks are blocked on or need assistance with.  </nowiki>
<rsquared> mizdebsk: What about the packages that are broken by the update?  What is there time line like?
 
<mizdebsk> rsquared: that is up to maintainers of dependant packages
<nowiki> <tstclair> ? </nowiki>
<mizdebsk> sometimes they are not fixed for multiple releases
 
<gil__> http://pastebin.com/xqed2aWv barchart-udt java7 problems
<nowiki> <mattf> i'm not blocked </nowiki>
* cwawak (~cwawak@redhat/cwawak) has left #fedora-meeting-1 ("Textual IRC Client: www.textualapp.com")
 
<mizdebsk> gil__: these gradle problems you mentioned are IMO minor
<nowiki> <gil__> lucene-analysis-morfologik https://bugzilla.redhat.com/show_bug.cgi?id=1002170 </nowiki>
* fabian_a has quit (Read error: Operation timed out)
 
<masta> rsquared: huh?
<nowiki> <tstclair> I'll follow up with willb tomorrow, but I know there were issues with scala and sbt. </nowiki>
<mizdebsk> the biggest 2 problems are version incompatibility and problems with bootstrapping
 
<mattf> masta, tab-completion fail
<nowiki> <gil__> solr core  https://bugzilla.redhat.com/show_bug.cgi?id=1005904 </nowiki>
* masta guesses that was a tab-completion thing
 
<masta> =)
<nowiki> <pmackinn> tstclair, https://bugzilla.redhat.com/show_bug.cgi?id=1010383 to start, fpc exception request filed </nowiki>
<tstclair> Another question which I've not been paying too much attention to, is there going to be a transition to 1.8 in the mid/near term?
 
<gil__> ah ok is for import latest littleproxy release with netty 4.x support(gradle)
<nowiki> <rsquared> hadoop-2.2.0 is currently blocked in rawhide by bz1023116. mizdebsk knows about it and is looking into it </nowiki>
<tstclair> Are there other items folks wanted to discuss...? gil__, mattf, mizdebsk, pmackinn, rsquared , besser82 ?
 
<mizdebsk> not really
<nowiki> <mattf> gil__, will you add those to the packages in review or to the review tracker? </nowiki>
<mattf> let's use the meeting bot next time
 
<mattf> that's it from me
<nowiki> <gil__> just the 1^ </nowiki>
<tstclair> +1
 
<pmackinn> no
<nowiki> <gil__> ops </nowiki>
<gil__> +1
 
* tim_1 (~chatzilla@c122-106-60-81.rivrw1.nsw.optusnet.com.au) has joined #fedora-meeting-1
<nowiki> <tstclair> I have no blockers. </nowiki>
<rsquared> I've got nothing more
 
<tstclair> kalev, I will try to summarize some of the items and I have a few open questions and I will email the list.
<nowiki> <tstclair> Item 3: Any new developments in the BIG-DATA ecosystem that are worth watching?  </nowiki>
<tstclair> Thanks everyone!
 
<nowiki> <tstclair> I've seen a lot of new ML coming into the works, no clear leaders. </nowiki>
 
<nowiki> <gil__> i have only a problem with solr contib module: </nowiki>
 
<nowiki> <tstclair> A lot of stuff based on Spark 0.8+ </nowiki>
 
<nowiki> <rsquared> HBase has a few dependencies awaiting review and I need to submit the jamon packages as well. </nowiki>
 
<nowiki> <rsquared> HBase itself is compiling but need eval of test suite </nowiki>
 
<nowiki> <mattf> at hadoop world this week i had a discussion and the topic of data nucleus came up. turns out it's something the community would like to get rid of. </nowiki>
 
<nowiki> <gil__> clustering require carrot2-mini-3.8.0 (this one depend with non-free mahout-math-0.6) </nowiki>
 
<nowiki> <rsquared> pmackinn: I'm sure you're glad to hear that about data nucleus </nowiki>
 
<nowiki> <pmackinn> i will eagerly follow upstream's lead in this regard </nowiki>
 
<nowiki> <mattf> ok, i'll give it a few days and see what shakes out </nowiki>
 
<nowiki> <tstclair> Is jline2 an issue for anyone, I have not dug in there. </nowiki>
 
<nowiki> <gil__> +1 </nowiki>
 
<nowiki> <mizdebsk> jline stuff should be no problem, there is either 1.x or 2.x to choose from </nowiki>
 
<nowiki> <rsquared> I'm not sure how much jline is used in hadoop, but I am expecting jline 2 to cause a problem.  Unsure if I'll need to rely on the compat package </nowiki>
 
<nowiki> <tstclair> mizdebsk, for now... I thought that there was the intention of deprecating. </nowiki>
 
<nowiki> <rsquared> tstclair: There's also netty and guava changes coming </nowiki>
 
<nowiki> <rsquared> I expect both to be an issue for hadoop </nowiki>
 
<nowiki> <mizdebsk> ideally, yes, i can't say much, but i've ported 2 packages to jline 2.10 and this was just a simple namespace change </nowiki>
 
<nowiki> <tstclair> do we need compat packages? </nowiki>
 
<nowiki> <gil__> netty breaks some thing also on gradle </nowiki>
 
<nowiki> <mizdebsk> for guava, i'll investigate </nowiki>
 
<nowiki> <mizdebsk> netty already has a compat package (netty31), netty will be updated to 4.x </nowiki>
 
<nowiki> <tstclair> rsquared, ^ </nowiki>
 
<nowiki> <gil__> for gradle netty31 is unusable </nowiki>
 
<nowiki> <rsquared> The maintainer of netty said he doesn't plan to maintain a compat package </nowiki>
 
<nowiki> <mizdebsk> rsquared: i am maintainer of netty and there already is a compat package </nowiki>
 
<nowiki> <tstclair> well don't they usually hang on until the deps transition? </nowiki>
 
<nowiki> <gil__> littleproxy is the library which caause the problem with gradle/netty </nowiki>
 
<nowiki> <rsquared> mizdebsk: who is Jon VanAlten then?  He sent the e-mails about the new netty and said he didn't plan to maintain a compat version </nowiki>
 
<nowiki> <mizdebsk> tstclair: of course, for reasonable time, but i cannot wait for months </nowiki>
 
<nowiki> <mizdebsk> rsquared: he is a comaintainer </nowiki>
 
<nowiki> <rsquared> hadoop uses the current netty (3.6).  Not sure if 3.1 will work.  Is there need/plan for a netty 3.6 compat package? </nowiki>
 
<nowiki> <gil__> newer littleproxy required some libraries wich dont work and dont compile under java7 </nowiki>
 
<nowiki> <tstclair> mizdebsk, what is a reasonable time? </nowiki>
 
<nowiki> <gil__> https://github.com/barchart/barchart-udt/ </nowiki>
 
<nowiki> <mizdebsk> tstclair: depends on the case, if i can see there are difficulties and there is some progress, i can wait </nowiki>
 
<nowiki> <gil__> Java wrapper for native C++ UDT protocol require also for build native libraries </nowiki>
 
<nowiki> <mizdebsk> otherwise i'm updating; maintainers of dependant packages can always fork the old version and convert it to compat package </nowiki>
 
<nowiki> <gil__> org.codeswarm:maven-nar-plugin:20121119 which is not compatible with our surefire version </nowiki>
 
<nowiki> <rsquared> gil__: Is gradle planning to be dropped? </nowiki>
 
<nowiki> * aeperezt (~aeperezt@fedora/aeperezt) has joined #fedora-meeting-1 </nowiki>
 
<nowiki> <gil__> yes </nowiki>
 
<nowiki> <gil__> rsquared: there are too many problems with gradle at the moment </nowiki>
 
<nowiki> <gil__> asm conflict </nowiki>
 
<nowiki> <gil__> newer eclipse aether apis </nowiki>
 
<nowiki> <rsquared> mizdebsk: What's the typical timeline line for updates like these?  If a package breaks in rawhide because of a dep update is there a problem so long as it's fixed by next release cycle? </nowiki>
 
<nowiki> <gil__> please mizdebsk,remember if i've forgotten something... </nowiki>
 
<nowiki> <mizdebsk> rsquared: according to the policy, announce 1 week in advance and then you can update </nowiki>
 
<nowiki> <mizdebsk> (that's why i wait with guava for next week) </nowiki>
 
<nowiki> <rsquared> mizdebsk: What about the packages that are broken by the update?  What is there time line like? </nowiki>
 
<nowiki> <mizdebsk> rsquared: that is up to maintainers of dependant packages </nowiki>
 
<nowiki> <mizdebsk> sometimes they are not fixed for multiple releases </nowiki>
 
<nowiki> <gil__> http://pastebin.com/xqed2aWv barchart-udt java7 problems </nowiki>
 
<nowiki> * cwawak (~cwawak@redhat/cwawak) has left #fedora-meeting-1 ("Textual IRC Client: www.textualapp.com") </nowiki>
 
<nowiki> <mizdebsk> gil__: these gradle problems you mentioned are IMO minor </nowiki>
 
<nowiki> * fabian_a has quit (Read error: Operation timed out) </nowiki>
 
<nowiki> <masta> rsquared: huh? </nowiki>
 
<nowiki> <mizdebsk> the biggest 2 problems are version incompatibility and problems with bootstrapping </nowiki>
 
<nowiki> <mattf> masta, tab-completion fail </nowiki>
 
<nowiki> * masta guesses that was a tab-completion thing </nowiki>
 
<nowiki> <masta> =) </nowiki>
 
<nowiki> <tstclair> Another question which I've not been paying too much attention to, is there going to be a transition to 1.8 in the mid/near term? </nowiki>
 
<nowiki> <gil__> ah ok is for import latest littleproxy release with netty 4.x support(gradle) </nowiki>
 
<nowiki> <tstclair> Are there other items folks wanted to discuss...? gil__, mattf, mizdebsk, pmackinn, rsquared , besser82 ? </nowiki>
 
<nowiki> <mizdebsk> not really </nowiki>
 
<nowiki> <mattf> let's use the meeting bot next time </nowiki>
 
<nowiki> <mattf> that's it from me </nowiki>
 
<nowiki> <tstclair> +1 </nowiki>
 
<nowiki> <pmackinn> no </nowiki>
 
<nowiki> <gil__> +1 </nowiki>
 
<nowiki> * tim_1 (~chatzilla@c122-106-60-81.rivrw1.nsw.optusnet.com.au) has joined #fedora-meeting-1 </nowiki>
 
<nowiki> <rsquared> I've got nothing more </nowiki>
 
<nowiki> <tstclair> kalev, I will try to summarize some of the items and I have a few open questions and I will email the list. </nowiki>
 
<nowiki> <tstclair> Thanks everyone! </nowiki>

Latest revision as of 16:50, 31 October 2013

<mattf> morning tim

masta mattf misc mjg59 mkolman mkolman_ muep

<tstclair> morning mattf

<mattf> who's here for the big data sig meeting?

* rsquared raises his hand

* orc_emac_ is now known as orc_fedo

* opuk (~kupo@h-1-5.a176.priv.bahnhof.se) has joined #fedora-meeting-1

* besser82 (~besser82@fedora/besser82) has joined #fedora-meeting-1

* gil__ (~quassel@151.60.14.100) has joined #fedora-meeting-1

<gil__> hi

<tstclair> Morning gil__

* mizdebsk (mizdebsk__@nat/redhat/x-qpgqpnidykcdkxrl) has joined #fedora-meeting-1

<tstclair> Morning Folks, I figured I would collect a list of agenda items: 1.) Status 2.) Issues / Blockers 3.) Feel free to add to the list.

<samkottler> tstclair: do we want to use meetbot?

<mattf> samkottler, good idea

<mattf> first meeting hiccups

<tstclair> samkottler, 1st time doing this..

<gil__> tstclair: +1

<samkottler> tstclair: hehe no worries :)

* suehle has quit (Ping timeout: 240 seconds)

<samkottler> tstclair: http://meetbot.debian.net/Manual.html is a good intro

<tstclair> I thought this channel was always recorded according to the docs.

<tstclair> I think we can figure out logistics for next round.

<tstclair> So current question: is https://fedoraproject.org/wiki/SIGs/bigdata/packaging up to date? And are there other items that folks thave been looking into?

* witlessb (~wb@cpc29-lewi15-2-0-cust72.2-4.cable.virginm.net) has joined #fedora-meeting-1

<tstclair> witlessb, <tstclair> So current question: is https://fedoraproject.org/wiki/SIGs/bigdata/packaging up to date? And are there other items that folks thave been looking into?

<mattf> gil__, i added solr to the packaging list when you announced it on the mailing list, fyi

<gil__> ok

<mattf> tstclair, it's up2date wrt what i'm working on

<tstclair> To give a brief synopsis: I've submitted curator and updated tachyon to build against the latest 0.4.0 branch. There were several issues in testing and I have to patches that hack around the issues and opened tickets upstream .

<tstclair> Are there any new developments that are not on the list?

<gil__> ... again some BR from finish to review....

<gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1002154

<tstclair> besser82, witlessb, rsquared, pmackinn any updates / issues ?

<gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1002170

<gil__> https://bugzilla.redhat.com/show_bug.cgi?id=1005904

<rsquared> Developments as in new bits being packaged or updates on existing packages?

<mizdebsk> i'm trying to find time to work on better Ivy integration, which should improve packaging of some applications (Scala stuff and others) </nowiki>

randomuser` rbergeron rdieter robyduck roshi rsquared

<tstclair> rsquared, both.

<besser82> tstclair: actually no issues / updates from me...

<pmackinn> tstclair, nothing here

<witlessb> tstclair: none from me

<tstclair> mizdebsk, I'm sure pmackinn and willb would be interested there.

<mattf> tstclair, did zookeeper get settled?

<rsquared> hadoop 2.2.0 in F20 has a minor issue with the hadoop-common testing jar/pom. It's misnamed. I've got a fix I can spin if it's blocking anyone.

<tstclair> I took ownership, I'm heavily dependent..

* msrb (msrb@nat/redhat/x-sllcvrytyozxcsgd) has joined #fedora-meeting-1

<tstclair> both tachyon and mesos require. </nowiki>

masta mattf misc mizdebsk mjg59 mkolman mkolman_ msrb muep

<tstclair> mattf, ^

<mattf> gotcha

<tstclair> are some of our friends from IBM in the audience?

<rsquared> masta: FYI, guava update coming in rawhide

<mizdebsk> i'll work on guava next week

<gil__> lucene-spatial https://bugzilla.redhat.com/show_bug.cgi?id=1002154

<tstclair> Second Item: Are there any issues folks are blocked on or need assistance with.

<tstclair> ?

<mattf> i'm not blocked

<gil__> lucene-analysis-morfologik https://bugzilla.redhat.com/show_bug.cgi?id=1002170

<tstclair> I'll follow up with willb tomorrow, but I know there were issues with scala and sbt.

<gil__> solr core https://bugzilla.redhat.com/show_bug.cgi?id=1005904

<pmackinn> tstclair, https://bugzilla.redhat.com/show_bug.cgi?id=1010383 to start, fpc exception request filed

<rsquared> hadoop-2.2.0 is currently blocked in rawhide by bz1023116. mizdebsk knows about it and is looking into it

<mattf> gil__, will you add those to the packages in review or to the review tracker?

<gil__> just the 1^

<gil__> ops

<tstclair> I have no blockers.

<tstclair> Item 3: Any new developments in the BIG-DATA ecosystem that are worth watching?

<tstclair> I've seen a lot of new ML coming into the works, no clear leaders.

<gil__> i have only a problem with solr contib module:

<tstclair> A lot of stuff based on Spark 0.8+

<rsquared> HBase has a few dependencies awaiting review and I need to submit the jamon packages as well.

<rsquared> HBase itself is compiling but need eval of test suite

<mattf> at hadoop world this week i had a discussion and the topic of data nucleus came up. turns out it's something the community would like to get rid of.

<gil__> clustering require carrot2-mini-3.8.0 (this one depend with non-free mahout-math-0.6)

<rsquared> pmackinn: I'm sure you're glad to hear that about data nucleus

<pmackinn> i will eagerly follow upstream's lead in this regard

<mattf> ok, i'll give it a few days and see what shakes out

<tstclair> Is jline2 an issue for anyone, I have not dug in there.

<gil__> +1

<mizdebsk> jline stuff should be no problem, there is either 1.x or 2.x to choose from

<rsquared> I'm not sure how much jline is used in hadoop, but I am expecting jline 2 to cause a problem. Unsure if I'll need to rely on the compat package

<tstclair> mizdebsk, for now... I thought that there was the intention of deprecating.

<rsquared> tstclair: There's also netty and guava changes coming

<rsquared> I expect both to be an issue for hadoop

<mizdebsk> ideally, yes, i can't say much, but i've ported 2 packages to jline 2.10 and this was just a simple namespace change

<tstclair> do we need compat packages?

<gil__> netty breaks some thing also on gradle

<mizdebsk> for guava, i'll investigate

<mizdebsk> netty already has a compat package (netty31), netty will be updated to 4.x

<tstclair> rsquared, ^

<gil__> for gradle netty31 is unusable

<rsquared> The maintainer of netty said he doesn't plan to maintain a compat package

<mizdebsk> rsquared: i am maintainer of netty and there already is a compat package

<tstclair> well don't they usually hang on until the deps transition?

<gil__> littleproxy is the library which caause the problem with gradle/netty

<rsquared> mizdebsk: who is Jon VanAlten then? He sent the e-mails about the new netty and said he didn't plan to maintain a compat version

<mizdebsk> tstclair: of course, for reasonable time, but i cannot wait for months

<mizdebsk> rsquared: he is a comaintainer

<rsquared> hadoop uses the current netty (3.6). Not sure if 3.1 will work. Is there need/plan for a netty 3.6 compat package?

<gil__> newer littleproxy required some libraries wich dont work and dont compile under java7

<tstclair> mizdebsk, what is a reasonable time?

<gil__> https://github.com/barchart/barchart-udt/

<mizdebsk> tstclair: depends on the case, if i can see there are difficulties and there is some progress, i can wait

<gil__> Java wrapper for native C++ UDT protocol require also for build native libraries

<mizdebsk> otherwise i'm updating; maintainers of dependant packages can always fork the old version and convert it to compat package

<gil__> org.codeswarm:maven-nar-plugin:20121119 which is not compatible with our surefire version

<rsquared> gil__: Is gradle planning to be dropped?

* aeperezt (~aeperezt@fedora/aeperezt) has joined #fedora-meeting-1

<gil__> yes

<gil__> rsquared: there are too many problems with gradle at the moment

<gil__> asm conflict

<gil__> newer eclipse aether apis

<rsquared> mizdebsk: What's the typical timeline line for updates like these? If a package breaks in rawhide because of a dep update is there a problem so long as it's fixed by next release cycle?

<gil__> please mizdebsk,remember if i've forgotten something...

<mizdebsk> rsquared: according to the policy, announce 1 week in advance and then you can update

<mizdebsk> (that's why i wait with guava for next week)

<rsquared> mizdebsk: What about the packages that are broken by the update? What is there time line like?

<mizdebsk> rsquared: that is up to maintainers of dependant packages

<mizdebsk> sometimes they are not fixed for multiple releases

<gil__> http://pastebin.com/xqed2aWv barchart-udt java7 problems

* cwawak (~cwawak@redhat/cwawak) has left #fedora-meeting-1 ("Textual IRC Client: www.textualapp.com")

<mizdebsk> gil__: these gradle problems you mentioned are IMO minor

* fabian_a has quit (Read error: Operation timed out)

<masta> rsquared: huh?

<mizdebsk> the biggest 2 problems are version incompatibility and problems with bootstrapping

<mattf> masta, tab-completion fail

* masta guesses that was a tab-completion thing

<masta> =)

<tstclair> Another question which I've not been paying too much attention to, is there going to be a transition to 1.8 in the mid/near term?

<gil__> ah ok is for import latest littleproxy release with netty 4.x support(gradle)

<tstclair> Are there other items folks wanted to discuss...? gil__, mattf, mizdebsk, pmackinn, rsquared , besser82 ?

<mizdebsk> not really

<mattf> let's use the meeting bot next time

<mattf> that's it from me

<tstclair> +1

<pmackinn> no

<gil__> +1

* tim_1 (~chatzilla@c122-106-60-81.rivrw1.nsw.optusnet.com.au) has joined #fedora-meeting-1

<rsquared> I've got nothing more

<tstclair> kalev, I will try to summarize some of the items and I have a few open questions and I will email the list.

<tstclair> Thanks everyone!