From Fedora Project Wiki

< SIGs‎ | bigdata

Revision as of 16:36, 31 October 2013 by Tstclair (talk | contribs) (Created page with "<mattf> who's here for the big data sig meeting? * rsquared raises his hand * orc_emac_ is now known as orc_fedo * opuk ( has joined #fedora-m...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

<mattf> who's here for the big data sig meeting?

  • rsquared raises his hand
  • orc_emac_ is now known as orc_fedo
  • opuk ( has joined #fedora-meeting-1
  • besser82 (~besser82@fedora/besser82) has joined #fedora-meeting-1
  • gil__ (~quassel@ has joined #fedora-meeting-1

<gil__> hi <tstclair> Morning gil__

  • mizdebsk (mizdebsk__@nat/redhat/x-qpgqpnidykcdkxrl) has joined #fedora-meeting-1

<tstclair> Morning Folks, I figured I would collect a list of agenda items: 1.) Status 2.) Issues / Blockers 3.) Feel free to add to the list. <samkottler> tstclair: do we want to use meetbot? <mattf> samkottler, good idea <mattf> first meeting hiccups <tstclair> samkottler, 1st time doing this.. <gil__> tstclair: +1 <samkottler> tstclair: hehe no worries :)

  • suehle has quit (Ping timeout: 240 seconds)

<samkottler> tstclair: is a good intro <tstclair> I thought this channel was always recorded according to the docs. <tstclair> I think we can figure out logistics for next round. <tstclair> So current question: is up to date? And are there other items that folks thave been looking into?

  • witlessb ( has joined #fedora-meeting-1

<tstclair> witlessb, <tstclair> So current question: is up to date? And are there other items that folks thave been looking into? <mattf> gil__, i added solr to the packaging list when you announced it on the mailing list, fyi <gil__> ok <mattf> tstclair, it's up2date wrt what i'm working on <tstclair> To give a brief synopsis: I've submitted curator and updated tachyon to build against the latest 0.4.0 branch. There were several issues in testing and I have to patches that hack around the issues and opened tickets upstream . <tstclair> Are there any new developments that are not on the list? <gil__> ... again some BR from finish to review.... <gil__> <tstclair> besser82, witlessb, rsquared, pmackinn any updates / issues ? <gil__> <gil__> <rsquared> Developments as in new bits being packaged or updates on existing packages? <mizdebsk> i'm trying to find time to work on better Ivy integration, which should improve packaging of some applications (Scala stuff and others)

randomuser` rbergeron rdieter robyduck roshi rsquared 

<tstclair> rsquared, both. <besser82> tstclair: actually no issues / updates from me... <pmackinn> tstclair, nothing here <witlessb> tstclair: none from me <tstclair> mizdebsk, I'm sure pmackinn and willb would be interested there. <mattf> tstclair, did zookeeper get settled? <rsquared> hadoop 2.2.0 in F20 has a minor issue with the hadoop-common testing jar/pom. It's misnamed. I've got a fix I can spin if it's blocking anyone. <tstclair> I took ownership, I'm heavily dependent..

  • msrb (msrb@nat/redhat/x-sllcvrytyozxcsgd) has joined #fedora-meeting-1

<tstclair> both tachyon and mesos require.

masta mattf misc mizdebsk mjg59 mkolman mkolman_ msrb muep 

<tstclair> mattf, ^ <mattf> gotcha <tstclair> are some of our friends from IBM in the audience? <rsquared> masta: FYI, guava update coming in rawhide <mizdebsk> i'll work on guava next week <gil__> lucene-spatial <tstclair> Second Item: Are there any issues folks are blocked on or need assistance with. <tstclair> ? <mattf> i'm not blocked <gil__> lucene-analysis-morfologik <tstclair> I'll follow up with willb tomorrow, but I know there were issues with scala and sbt. <gil__> solr core <pmackinn> tstclair, to start, fpc exception request filed <rsquared> hadoop-2.2.0 is currently blocked in rawhide by bz1023116. mizdebsk knows about it and is looking into it <mattf> gil__, will you add those to the packages in review or to the review tracker? <gil__> just the 1^ <gil__> ops <tstclair> I have no blockers. <tstclair> Item 3: Any new developments in the BIG-DATA ecosystem that are worth watching? <tstclair> I've seen a lot of new ML coming into the works, no clear leaders. <gil__> i have only a problem with solr contib module: <tstclair> A lot of stuff based on Spark 0.8+ <rsquared> HBase has a few dependencies awaiting review and I need to submit the jamon packages as well. <rsquared> HBase itself is compiling but need eval of test suite <mattf> at hadoop world this week i had a discussion and the topic of data nucleus came up. turns out it's something the community would like to get rid of. <gil__> clustering require carrot2-mini-3.8.0 (this one depend with non-free mahout-math-0.6) <rsquared> pmackinn: I'm sure you're glad to hear that about data nucleus <pmackinn> i will eagerly follow upstream's lead in this regard <mattf> ok, i'll give it a few days and see what shakes out <tstclair> Is jline2 an issue for anyone, I have not dug in there. <gil__> +1 <mizdebsk> jline stuff should be no problem, there is either 1.x or 2.x to choose from <rsquared> I'm not sure how much jline is used in hadoop, but I am expecting jline 2 to cause a problem. Unsure if I'll need to rely on the compat package <tstclair> mizdebsk, for now... I thought that there was the intention of deprecating. <rsquared> tstclair: There's also netty and guava changes coming <rsquared> I expect both to be an issue for hadoop <mizdebsk> ideally, yes, i can't say much, but i've ported 2 packages to jline 2.10 and this was just a simple namespace change <tstclair> do we need compat packages? <gil__> netty breaks some thing also on gradle <mizdebsk> for guava, i'll investigate <mizdebsk> netty already has a compat package (netty31), netty will be updated to 4.x <tstclair> rsquared, ^ <gil__> for gradle netty31 is unusable <rsquared> The maintainer of netty said he doesn't plan to maintain a compat package <mizdebsk> rsquared: i am maintainer of netty and there already is a compat package <tstclair> well don't they usually hang on until the deps transition? <gil__> littleproxy is the library which caause the problem with gradle/netty <rsquared> mizdebsk: who is Jon VanAlten then? He sent the e-mails about the new netty and said he didn't plan to maintain a compat version <mizdebsk> tstclair: of course, for reasonable time, but i cannot wait for months <mizdebsk> rsquared: he is a comaintainer <rsquared> hadoop uses the current netty (3.6). Not sure if 3.1 will work. Is there need/plan for a netty 3.6 compat package? <gil__> newer littleproxy required some libraries wich dont work and dont compile under java7 <tstclair> mizdebsk, what is a reasonable time? <gil__> <mizdebsk> tstclair: depends on the case, if i can see there are difficulties and there is some progress, i can wait <gil__> Java wrapper for native C++ UDT protocol require also for build native libraries <mizdebsk> otherwise i'm updating; maintainers of dependant packages can always fork the old version and convert it to compat package <gil__> org.codeswarm:maven-nar-plugin:20121119 which is not compatible with our surefire version <rsquared> gil__: Is gradle planning to be dropped?

  • aeperezt (~aeperezt@fedora/aeperezt) has joined #fedora-meeting-1

<gil__> yes <gil__> rsquared: there are too many problems with gradle at the moment <gil__> asm conflict <gil__> newer eclipse aether apis <rsquared> mizdebsk: What's the typical timeline line for updates like these? If a package breaks in rawhide because of a dep update is there a problem so long as it's fixed by next release cycle? <gil__> please mizdebsk,remember if i've forgotten something... <mizdebsk> rsquared: according to the policy, announce 1 week in advance and then you can update <mizdebsk> (that's why i wait with guava for next week) <rsquared> mizdebsk: What about the packages that are broken by the update? What is there time line like? <mizdebsk> rsquared: that is up to maintainers of dependant packages <mizdebsk> sometimes they are not fixed for multiple releases <gil__> barchart-udt java7 problems

  • cwawak (~cwawak@redhat/cwawak) has left #fedora-meeting-1 ("Textual IRC Client:")

<mizdebsk> gil__: these gradle problems you mentioned are IMO minor

  • fabian_a has quit (Read error: Operation timed out)

<masta> rsquared: huh? <mizdebsk> the biggest 2 problems are version incompatibility and problems with bootstrapping <mattf> masta, tab-completion fail

  • masta guesses that was a tab-completion thing

<masta> =) <tstclair> Another question which I've not been paying too much attention to, is there going to be a transition to 1.8 in the mid/near term? <gil__> ah ok is for import latest littleproxy release with netty 4.x support(gradle) <tstclair> Are there other items folks wanted to discuss...? gil__, mattf, mizdebsk, pmackinn, rsquared , besser82 ? <mizdebsk> not really <mattf> let's use the meeting bot next time <mattf> that's it from me <tstclair> +1 <pmackinn> no <gil__> +1

  • tim_1 ( has joined #fedora-meeting-1

<rsquared> I've got nothing more <tstclair> kalev, I will try to summarize some of the items and I have a few open questions and I will email the list. <tstclair> Thanks everyone!