git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Committee to Sort through CCC Presentation Submissions


Will – What do you think? With only 26 presentations, do you think it would be reasonable to just ask each reviewer to review each one? One time that I was on one of these panels a couple years ago, we each reviewed the roughly dozen presentations that were submitted. Of course, people may not be able to spend that amount of time on this.

> On Apr 5, 2018, at 7:14 PM, Ron Wheeler <rwheeler@xxxxxxxxxxxxxxxxxxxxx> wrote:
> 
> We still need to manage the review process and make sure that it is adequately staffed.
> 
> The allocation of presentations to reviewers has to be managed to be sure that the reviewers have the support that they need to do a proper review and that the reviews get done.
> 
> Ron
> 
> 
>> On 05/04/2018 11:45 AM, Tutkowski, Mike wrote:
>> Perfect…then, unless anyone has other opinions they’d like to share on the topic, let’s follow that approach.
>> 
>> On 4/5/18, 9:43 AM, "Rafael Weingärtner" <rafaelweingartner@xxxxxxxxx> wrote:
>> 
>>     That is exactly it.
>>          On Thu, Apr 5, 2018 at 12:37 PM, Tutkowski, Mike <Mike.Tutkowski@xxxxxxxxxx>
>>     wrote:
>>          > Hi Rafael,
>>     >
>>     > I think as long as we (the CloudStack Community) have the final say on how
>>     > we fill our allotted slots in the CloudStack track of ApacheCon in
>>     > Montreal, then it’s perfectly fine for us to leverage Apache’s normal
>>     > review process to gather all the feedback from the larger Apache Community.
>>     >
>>     > As you say, we could wait for the feedback to come in via that mechanism
>>     > and then, as per Will’s earlier comments, we could advertise on our users@
>>     > and dev@ mailing lists when we plan to get together for a call and make
>>     > final decisions on the CFP.
>>     >
>>     > Is that, in fact, what you were thinking, Rafael?
>>     >
>>     > Talk to you soon,
>>     > Mike
>>     >
>>     > On 4/4/18, 2:58 PM, "Rafael Weingärtner" <rafaelweingartner@xxxxxxxxx>
>>     > wrote:
>>     >
>>     >     I think everybody that “raised their hands here” already signed up to
>>     >     review.
>>     >
>>     >     Mike, what about if we only gathered the reviews from Apache main
>>     > review
>>     >     system, and then we use that to decide which presentations will get in
>>     >     CloudStack tracks? Then, we reduce the work on our side (we also remove
>>     >     bias…). I do believe that the review from other peers from Apache
>>     > community
>>     >     (even the one outside from our small community) will be fair and
>>     > technical
>>     >     (meaning, without passion and or favoritism).
>>     >
>>     >     Having said that, I think we only need a small group of PMCs to gather
>>     > the
>>     >     results and out of the best ranked proposals, we pick the ones to our
>>     >     tracks.
>>     >
>>     >     What do you (Mike) and others think?
>>     >
>>     >
>>     >     On Tue, Apr 3, 2018 at 5:07 PM, Tutkowski, Mike <
>>     > Mike.Tutkowski@xxxxxxxxxx>
>>     >     wrote:
>>     >
>>     >     > Hi Ron,
>>     >     >
>>     >     > I don’t actually have insight into how many people have currently
>>     > signed
>>     >     > up online to be CFP reviewers for ApacheCon. At present, I’m only
>>     > aware of
>>     >     > those who have responded to this e-mail chain.
>>     >     >
>>     >     > We should be able to find out more in the coming weeks. We’re still
>>     > quite
>>     >     > early in the process.
>>     >     >
>>     >     > Thanks for your feedback,
>>     >     > Mike
>>     >     >
>>     >     > On 4/1/18, 9:18 AM, "Ron Wheeler" <rwheeler@xxxxxxxxxxxxxxxxxxxxx>
>>     > wrote:
>>     >     >
>>     >     >     How many people have signed up to be reviewers?
>>     >     >
>>     >     >     I don't think that scheduling is part of the review process and
>>     > that
>>     >     > can
>>     >     >     be done by the person/team "organizing" ApacheCon on behalf of
>>     > the PMC.
>>     >     >
>>     >     >     To me review is looking at content for
>>     >     >     - relevance
>>     >     >     - quality of the presentations (suggest fixes to content,
>>     > English,
>>     >     >     graphics, etc.)
>>     >     >     This should result in a consensus score
>>     >     >     - Perfect - ready for prime time
>>     >     >     - Needs minor changes as documented by the reviewers
>>     >     >     - Great topic but needs more work - perhaps a reviewer could
>>     > volunteer
>>     >     >     to work with the presenter to get it ready if chosen
>>     >     >     - Not recommended for topic or content reasons
>>     >     >
>>     >     >     The reviewers could also make non-binding recommendations about
>>     > the
>>     >     >     balance between topics - marketing(why Cloudstack),
>>     >     >     Operations/implementation, Technical details, Roadmap, etc.
>>     > based on
>>     >     >     what they have seen.
>>     >     >
>>     >     >     This should be used by the organizers to make the choices and
>>     > organize
>>     >     >     the program.
>>     >     >     The organizers have the final say on the choice of presentations
>>     > and
>>     >     >     schedule
>>     >     >
>>     >     >     Reviewers are there to help the process not control it.
>>     >     >
>>     >     >     I would be worried that you do not have enough reviewers rather
>>     > than
>>     >     > too
>>     >     >     many.
>>     >     >     Then the work falls on the PMC and organizers.
>>     >     >
>>     >     >     When planning meetings, I would recommend that you clearly
>>     > separate the
>>     >     >     roles and only invite the reviewers to the meetings about
>>     > review. Get
>>     >     >     the list of presentation to present to the reviewers and decide
>>     > if
>>     >     > there
>>     >     >     are any instructions that you want to give to reviewers.
>>     >     >     I would recommend that you keep the organizing group small.
>>     > Membership
>>     >     >     should be set by the PMC and should be people that are committed
>>     > to the
>>     >     >     ApacheCon project and have the time. The committee can request
>>     > help for
>>     >     >     specific tasks from others in the community who are not on the
>>     >     > committee.
>>     >     >
>>     >     >     I would also recommend that organizers do not do reviews. They
>>     > should
>>     >     >     read the finalists but if they do reviews, there may be a
>>     > suggestion of
>>     >     >     favouring presentations that they reviewed. It also ensures that
>>     > the
>>     >     >     organizers are not getting heat from rejected presenters - "it
>>     > is the
>>     >     >     reviewers fault you did not get selected".
>>     >     >
>>     >     >     My advice is to get as many reviewers as you can so that no one
>>     > is
>>     >     >     essential and each reviewer has a limited number of
>>     > presentations to
>>     >     >     review but each presentation gets reviewed by multiple people.
>>     > Also
>>     >     > bear
>>     >     >     in mind that not all reviewers have the same ability to review
>>     > each
>>     >     >     presentation.
>>     >     >     Reviews should be anonymous and only the summary comments given
>>     > to the
>>     >     >     presenter. Reviewers of a presentation should be able to discuss
>>     > the
>>     >     >     presentation during the review to make sure that reviewers do
>>     > not feel
>>     >     >     isolated or get lost when they hit content that they don't
>>     > understand
>>     >     > fully.
>>     >     >
>>     >     >
>>     >     >
>>     >     >     Ron
>>     >     >
>>     >     >
>>     >     >     On 01/04/2018 12:20 AM, Tutkowski, Mike wrote:
>>     >     >     > Thanks for the feedback, Will!
>>     >     >     >
>>     >     >     > I agree with the approach you outlined.
>>     >     >     >
>>     >     >     > Thanks for being so involved in the process! Let’s chat with
>>     > Giles
>>     >     > once he’s back to see if we can get your questions answered.
>>     >     >     >
>>     >     >     >> On Mar 31, 2018, at 10:14 PM, Will Stevens <
>>     > wstevens@xxxxxxxxxxxx>
>>     >     > wrote:
>>     >     >     >>
>>     >     >     >> In the past the committee was chosen as a relatively small
>>     > group in
>>     >     > order
>>     >     >     >> to make it easier to manage feedback.  In order to make it
>>     > fair to
>>     >     > everyone
>>     >     >     >> in the community, I would suggest that instead of doing it
>>     > with a
>>     >     > small
>>     >     >     >> group, we do it out in the open on a scheduled call.
>>     >     >     >>
>>     >     >     >> We will have to get a list of the talks that are CloudStack
>>     >     > specific from
>>     >     >     >> ApacheCon, but that should be possible.
>>     >     >     >>
>>     >     >     >> Once we have the talks selected, then a smaller number of us
>>     > can
>>     >     > work on
>>     >     >     >> setting up the actual ordering and the details.
>>     >     >     >>
>>     >     >     >> I have been quite involved so far.  Giles and I have been
>>     >     > organizing the
>>     >     >     >> sponsors, website and dealing with ApacheCon so far.
>>     > Obviously,
>>     >     > Mike is
>>     >     >     >> also working on this as well.
>>     >     >     >>
>>     >     >     >> I think we are headed in the right direction on this.
>>     >     >     >>
>>     >     >     >> Cheers,
>>     >     >     >>
>>     >     >     >> Will
>>     >     >     >>
>>     >     >     >> On Mar 31, 2018 11:49 PM, "Tutkowski, Mike" <
>>     >     > Mike.Tutkowski@xxxxxxxxxx>
>>     >     >     >> wrote:
>>     >     >     >>
>>     >     >     >> Hi Ron,
>>     >     >     >>
>>     >     >     >> I am definitely open to working this however makes the most
>>     > sense.
>>     >     >     >>
>>     >     >     >> It looks like Will’s e-mail indicates that the process I
>>     > suggested
>>     >     > has been
>>     >     >     >> followed in the past (which is how I recall, as well).
>>     >     >     >>
>>     >     >     >> Let’s make sure I understood Will correctly.
>>     >     >     >>
>>     >     >     >> Will – Are you, in fact, indicating that what I was
>>     > suggesting is
>>     >     > how we
>>     >     >     >> have reviewed the CFP in the past? If so, are you able to
>>     > address
>>     >     > Ron’s
>>     >     >     >> concerns?
>>     >     >     >>
>>     >     >     >> Also, Will – I am not sure about a hackathon. Let’s chat with
>>     > Giles
>>     >     > once
>>     >     >     >> he’s back from vacation since he’s been the most involved with
>>     >     > organizing
>>     >     >     >> the CloudStack track within ApacheCon.
>>     >     >     >>
>>     >     >     >> Thanks!
>>     >     >     >>
>>     >     >     >> Mike
>>     >     >     >>
>>     >     >     >>
>>     >     >     >> On 3/31/18, 9:00 PM, "Ron Wheeler" <
>>     > rwheeler@xxxxxxxxxxxxxxxxxxxxx>
>>     >     > wrote:
>>     >     >     >>
>>     >     >     >>     I am not sure about your concern in that case.
>>     >     >     >>     I am not sure why people not interested in Cloudstack
>>     > would
>>     >     > volunteer as
>>     >     >     >>     reviewers and want to pick bad presentations.
>>     >     >     >>
>>     >     >     >>     I would be more worried that there are not enough good
>>     >     > presentations
>>     >     >     >>     proposed rather than some meritorious presentation will
>>     > get
>>     >     > rejected due
>>     >     >     >>     to "outsiders" voting it down in favour of less useful
>>     >     > presentations.
>>     >     >     >>
>>     >     >     >>     It may be tricky to get balance if that means taking "bad"
>>     >     > proposals
>>     >     >     >>     that can not be fixed that cover topics that are in areas
>>     > that
>>     >     > are not
>>     >     >     >>     otherwise covered at the expense of great presentations
>>     > that
>>     >     > are in
>>     >     >     >>     areas with many choices.
>>     >     >     >>
>>     >     >     >>     We should wait to see how many presentations have to be
>>     >     > rejected and the
>>     >     >     >>     number of reviewers before getting too exercised over the
>>     >     > loyalty of
>>     >     >     >>     reviewers.
>>     >     >     >>
>>     >     >     >>     Getting more reviewers is likely the most effective way
>>     > to see
>>     >     > that a
>>     >     >     >>     wider range of topics is covered.
>>     >     >     >>
>>     >     >     >>     Ron
>>     >     >     >>
>>     >     >     >>>     On 31/03/2018 7:15 PM, Tutkowski, Mike wrote:
>>     >     >     >>> Hi Ron,
>>     >     >     >>>
>>     >     >     >>>  From what I understand, the CloudStack proposals will be
>>     > mixed in
>>     >     >     >> with all of the ApacheCon proposals.
>>     >     >     >>> In the past when I’ve participated in these CloudStack
>>     > panels to
>>     >     >     >> review proposals, we had to compare each proposal against the
>>     >     > others to
>>     >     >     >> arrive at a balance of topics (i.e. not all networking
>>     > focused, not
>>     >     > all
>>     >     >     >> XenServer focused, etc.) and to suggest improvements for
>>     > proposals
>>     >     > that we
>>     >     >     >> did not accept for other reasons.
>>     >     >     >>>  From what I understand (but Giles can comment further on
>>     > this), we
>>     >     >     >> have a track at ApacheCon and will need to fill it with X
>>     > number of
>>     >     >     >> presentations. To do this, it seems like a CloudStack-focused
>>     > panel
>>     >     > would
>>     >     >     >> be a good approach, but I am definitely open to another
>>     > approach.
>>     >     > We don’t
>>     >     >     >> want to exclude anyone (in or out of the CloudStack
>>     > Community) who
>>     >     > might
>>     >     >     >> like to provide input. Anyone who is interested would, of
>>     > course,
>>     >     > be free
>>     >     >     >> to join us in combing through the proposals.
>>     >     >     >>> We don’t need to get started on this right away. The CFP just
>>     >     > closed
>>     >     >     >> yesterday. Let’s wait for feedback from Giles (who is
>>     > currently on
>>     >     >     >> vacation) and go from there.
>>     >     >     >>> Thanks!
>>     >     >     >>> Mike
>>     >     >     >>>
>>     >     >     >>> On 3/31/18, 6:59 AM, "Ron Wheeler" <
>>     > rwheeler@xxxxxxxxxxxxxxxxxxxxx
>>     >     > >
>>     >     >     >> wrote:
>>     >     >     >>>      Is this a real concern?
>>     >     >     >>>      Why would a large number of Apache contributors who are
>>     > not
>>     >     >     >> interested
>>     >     >     >>>      in Cloudstack (enough to outvote those "part of the
>>     > Cloudstack
>>     >     >     >>>      community") get involved as reviewers
>>     >     >     >>>
>>     >     >     >>>      Reviewing involves some commitment of time so I am hard
>>     >     > pressed
>>     >     >     >> to guess
>>     >     >     >>>      why some Apache contributor would volunteer to do the
>>     > work in
>>     >     >     >> order to
>>     >     >     >>>      veto a presentation that they have not yet seen or have
>>     > no
>>     >     >     >> interest in
>>     >     >     >>>      seeing.
>>     >     >     >>>
>>     >     >     >>>      Are we guaranteed a fixed number of hours of
>>     > presentations or
>>     >     > is
>>     >     >     >> the
>>     >     >     >>>      review process part of the allocation of overall time?
>>     >     >     >>>
>>     >     >     >>>      On what basis can some group veto a presentation?
>>     >     >     >>>      That would seem to be a very strong action and I would
>>     > hope
>>     >     > that
>>     >     >     >> it
>>     >     >     >>>      requires a strong reason.
>>     >     >     >>>
>>     >     >     >>>      OTOH if a large??? number of Apache contributors
>>     > (regardless
>>     >     > of
>>     >     >     >> their
>>     >     >     >>>      affiliation) say that a presentation has serious issues
>>     > or
>>     >     > very
>>     >     >     >> limited
>>     >     >     >>>      interest, that would seem to be a red flag that the
>>     >     > presentation
>>     >     >     >>>      requires improvement or needs to be dropped in favour of
>>     >     > another
>>     >     >     >>>      Cloudstack presentation, if it can not be fixed.
>>     >     >     >>>
>>     >     >     >>>      We should also be aware that this is an opportunity to
>>     >     > "market"
>>     >     >     >>>      Cloudstack to the broader Apache community.
>>     >     >     >>>      Outside reviewers might have valuable input into how
>>     >     >     >> presentations can
>>     >     >     >>>      attract new adopters or be clearer to the broader DevOps
>>     >     >     >> community.
>>     >     >     >>>      We also need to remember that we do have an active
>>     > community
>>     >     > and
>>     >     >     >> other
>>     >     >     >>>      opportunities during the year to present presentations
>>     > that do
>>     >     >     >> not get
>>     >     >     >>>      selected for this conference.
>>     >     >     >>>
>>     >     >     >>>      If their is a real fear that a lot of "outsiders" are
>>     > going to
>>     >     >     >> disrupt
>>     >     >     >>>      the review process, a more reasonable response would
>>     > seem to
>>     >     > be
>>     >     >     >> to get
>>     >     >     >>>      more reviewers from the community.
>>     >     >     >>>
>>     >     >     >>>      I have volunteered already.
>>     >     >     >>>
>>     >     >     >>>      Ron
>>     >     >     >>>
>>     >     >     >>>>      On 30/03/2018 11:11 PM, Tutkowski, Mike wrote:
>>     >     >     >>>> Hi Rafael,
>>     >     >     >>>>
>>     >     >     >>>> It’s a little bit tricky in our particular situation. Allow
>>     > me
>>     >     >     >> to explain:
>>     >     >     >>>> As you are likely aware, the CloudStack Collaboration
>>     >     >     >> Conference will be held as a track in the larger ApacheCon
>>     >     > conference in
>>     >     >     >> Montreal this coming September.
>>     >     >     >>>> It is true, as you say, that anyone who wishes to do so can
>>     >     >     >> contribute to reviewing the CFP for ApacheCon.
>>     >     >     >>>> What is a bit of a concern, however, is that we might get
>>     >     >     >> certain CloudStack CFP proposals vetoed by people who are
>>     > not, per
>>     >     > se, a
>>     >     >     >> part of our community.
>>     >     >     >>>> That being the case, I have contacted the organizers for
>>     >     >     >> ApacheCon to see if there is some way we can section off the
>>     >     > CloudStack CFP
>>     >     >     >> from the larger ApacheCon CFP for review purposes.
>>     >     >     >>>> Assuming we can do this, the panel that I am proposing here
>>     >     >     >> would handle this review task.
>>     >     >     >>>> I hope that helps clarify the situation.
>>     >     >     >>>>
>>     >     >     >>>> Thanks!
>>     >     >     >>>> Mike
>>     >     >     >>>>
>>     >     >     >>>> On 3/30/18, 8:38 AM, "Rafael Weingärtner" <
>>     >     >     >> rafaelweingartner@xxxxxxxxx> wrote:
>>     >     >     >>>>      Are we going to have a separated review process?
>>     >     >     >>>>
>>     >     >     >>>>      I thought anybody could go here [1] and apply for a
>>     >     >     >> reviewer position and
>>     >     >     >>>>      start reviewing. Well, that is what I did. I have
>>     > already
>>     >     >     >> reviewed some
>>     >     >     >>>>      CloudStack proposals (of course I did not review
>>     > mines).
>>     >     >     >> After asking to
>>     >     >     >>>>      review presentations, Rich has giving me access to the
>>     >     >     >> system. I thought
>>     >     >     >>>>      everybody interest in helping was going to do the same.
>>     >     >     >>>>
>>     >     >     >>>>      [1]
>>     >     >     >> https://cfp.apachecon.com/conference.html?apachecon-
>>     >     > north-america-2018
>>     >     >     >>>>
>>     >     >     >>>>      On Wed, Mar 28, 2018 at 4:05 AM, Swen - swen.io <
>>     >     >     >> me@xxxxxxx> wrote:
>>     >     >     >>>>> Hi Mike,
>>     >     >     >>>>>
>>     >     >     >>>>> congrats!
>>     >     >     >>>>>
>>     >     >     >>>>> I can help sort through presentations.
>>     >     >     >>>>>
>>     >     >     >>>>> Best regards,
>>     >     >     >>>>> Swen
>>     >     >     >>>>>
>>     >     >     >>>>> -----Ursprüngliche Nachricht-----
>>     >     >     >>>>> Von: Tutkowski, Mike [mailto:Mike.Tutkowski@xxxxxxxxxx]
>>     >     >     >>>>> Gesendet: Dienstag, 27. März 2018 21:40
>>     >     >     >>>>> An: dev@xxxxxxxxxxxxxxxxxxxxx;
>>     >     >     >> users@xxxxxxxxxxxxxxxxxxxxx
>>     >     >     >>>>> Betreff: Committee to Sort through CCC Presentation
>>     >     >     >> Submissions
>>     >     >     >>>>> Hi everyone,
>>     >     >     >>>>>
>>     >     >     >>>>> As you may be aware, this coming September in Montreal,
>>     >     >     >> the CloudStack
>>     >     >     >>>>> Community will be hosting the CloudStack Collaboration
>>     >     >     >> Conference:
>>     >     >     >>>>> http://ca.cloudstackcollab.org/
>>     >     >     >>>>>
>>     >     >     >>>>> Even though the event is six months away, we are on a
>>     >     >     >> tight schedule with
>>     >     >     >>>>> regards to the Call For Participation (CFP):
>>     >     >     >>>>>
>>     >     >     >>>>> https://www.apachecon.com/acna18/schedule.html
>>     >     >     >>>>>
>>     >     >     >>>>> If you are interested in submitting a talk, please do
>>     >     >     >> so before March 30th.
>>     >     >     >>>>> That being said, as usual, we will have need of a small
>>     >     >     >> committee to sort
>>     >     >     >>>>> through these presentation submissions.
>>     >     >     >>>>>
>>     >     >     >>>>> If you are interested in helping out in this process,
>>     >     >     >> please reply to this
>>     >     >     >>>>> message.
>>     >     >     >>>>>
>>     >     >     >>>>> Thanks!
>>     >     >     >>>>> Mike
>>     >     >     >>>>>
>>     >     >     >>>>>
>>     >     >     >>>>>
>>     >     >     >>>>
>>     >     >     >>>>      --
>>     >     >     >>>>      Rafael Weingärtner
>>     >     >     >>>>
>>     >     >     >>>>
>>     >     >     >>>
>>     >     >
>>     >     >     --
>>     >     >     Ron Wheeler
>>     >     >     President
>>     >     >     Artifact Software Inc
>>     >     >     email: rwheeler@xxxxxxxxxxxxxxxxxxxxx
>>     >     >     skype: ronaldmwheeler
>>     >     >     phone: 866-970-2435, ext 102
>>     >     >
>>     >     >
>>     >     >
>>     >     >
>>     >
>>     >
>>     >     --
>>     >     Rafael Weingärtner
>>     >
>>     >
>>     >
>>               --
>>     Rafael Weingärtner
>>     
> 
> -- 
> Ron Wheeler
> President
> Artifact Software Inc
> email: rwheeler@xxxxxxxxxxxxxxxxxxxxx
> skype: ronaldmwheeler
> phone: 866-970-2435, ext 102
> 


( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-cloudstack-development/msg07216.html on line 621
Call Stack
#TimeMemoryFunctionLocation
10.0012397512{main}( ).../msg07216.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-cloudstack-development/msg07216.html on line 621
Call Stack
#TimeMemoryFunctionLocation
10.0012397512{main}( ).../msg07216.html:0