git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: RFC. Marvin tests fail


Hello Boris,
During the troubleshooting, I have found that the problem is connected with
my host environment, so I decided to build the Dockerfile which is for
"simulator & client" running altogether.

The Dockerfile itself is, for example, is here:
https://pastebin.com/raw/ZVjtHX7F

The Dockerfile is redesigned from the original simulator the way to be fast
built for QA E2E tests from the point of view:
- java code changes very rarely;
- marvin code changes rarely;
- tests change often.

After being built it can be used like:
Simulator run: docker run --name sim -v my-tmp:/tmp -it --rm simulator:4.11
Tests  run: docker exec -it sim bash /root/docker_run_tests.sh
<advanced|basic>

If the community is interested in then I can supersede original
tools/docker/Dockerfile with that one.

Also, I added .dockerignore to avoid Docker cache invalidation.

So, to sum up, this Dockerfile is for easy Marvin development and testing
in the isolated, stable environment.

Give me your feedback, please. All changes are here:
https://github.com/apache/cloudstack/pull/3012/files



пт, 9 нояб. 2018 г. в 2:07, Boris Stoyanov <boris.stoyanov@xxxxxxxxxxxxx>:

> you could run it with a debugger in PyCharm for example and figure out if
> there’s anything wrong with the code. Please note that these tests are
> successful with KVM, VMware and Xen.
>
> The logs you’ve shared are just results, there’s a failed_plus_exceptions
> file in each test file log, where you could find more detailed info and
> stack trace.
>
> Boris.
>
>
> boris.stoyanov@xxxxxxxxxxxxx
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
> > On 8 Nov 2018, at 2:37, Ivan Kudryavtsev <kudryavtsev_ia@xxxxxxxxx>
> wrote:
> >
> > Looks like the reason for the failure is in python code, not tests.
> Someone
> > broke tests.
> >
> > ср, 7 нояб. 2018 г. в 19:29, Ivan Kudryavtsev <kudryavtsev_ia@xxxxxxxxx
> >:
> >
> >> Well, when I run outside of docker. I see another behavior:
> >> Those OOB tests work well, but I meet errors in other pieces of code:
> >>
> >> *ivan@notebook:/tmp/MarvinLogs/Nov_07_2018_18_53_35_8SWDQD$ grep -P
> >> 'EXCEPTION|FAIL' results.txt *
> >> Tests ha enable/disable feature at cluster and zone level ... ===
> >> TestName: test_ha_configure_enabledisable_across_clusterzones | Status :
> >> EXCEPTION ===
> >> Tests ha resource ownership expiry across multi-mgmt server ... ===
> >> TestName: test_ha_multiple_mgmt_server_ownership | Status : EXCEPTION
> ===
> >> Tests ha FSM transitions for valid healthy host ... === TestName:
> >> test_ha_verify_fsm_available | Status : EXCEPTION ===
> >> Tests ha FSM transitions leading to degraded state ... === TestName:
> >> test_ha_verify_fsm_degraded | Status : EXCEPTION ===
> >> Tests ha FSM transitions for failures leading to fenced state ... ===
> >> TestName: test_ha_verify_fsm_fenced | Status : EXCEPTION ===
> >> Tests ha FSM transitions leading to recovering ... === TestName:
> >> test_ha_verify_fsm_recovering | Status : EXCEPTION ===
> >> === TestName: test_list_zones_metrics | Status : EXCEPTION ===
> >> Tests out-of-band management ownership expiry across multi-mgmt server
> ...
> >> === TestName: test_oobm_multiple_mgmt_server_ownership | Status :
> FAILED ===
> >> FAIL
> >> === TestName: test_ha_configure_enabledisable_across_clusterzones |
> Status
> >> : EXCEPTION ===
> >> === TestName: test_ha_multiple_mgmt_server_ownership | Status :
> EXCEPTION
> >> ===
> >> === TestName: test_ha_verify_fsm_available | Status : EXCEPTION ===
> >> === TestName: test_ha_verify_fsm_degraded | Status : EXCEPTION ===
> >> === TestName: test_ha_verify_fsm_fenced | Status : EXCEPTION ===
> >> === TestName: test_ha_verify_fsm_recovering | Status : EXCEPTION ===
> >> FAIL: Tests out-of-band management ownership expiry across multi-mgmt
> >> server
> >> === TestName: test_oobm_multiple_mgmt_server_ownership | Status : FAILED
> >> ===
> >> FAILED (SKIP=13, errors=6, failures=1)
> >>
> >> I pasted errors to Pastebin: https://pastebin.com/tXw6mRk7
> >>
> >>
> >>
> >>
> >> ср, 7 нояб. 2018 г. в 14:05, Boris Stoyanov <
> boris.stoyanov@xxxxxxxxxxxxx
> >>> :
> >>
> >>> Hi Ivan,
> >>> I guess you’re referring to Out of Band management? I think there was a
> >>> simulator provider for those tests and they are reported as passing
> with
> >>> latest KVM test run.
> >>>
> >>> Can you share any logs/exceptions? I don’t see anything wrong with your
> >>> setup on a first glance btw.
> >>>
> >>> Bobby.
> >>>
> >>>
> >>> boris.stoyanov@xxxxxxxxxxxxx
> >>> www.shapeblue.com
> >>> Amadeus House, Floral Street, London  WC2E 9DPUK
> >>> @shapeblue
> >>>
> >>>
> >>>
> >>>> On 7 Nov 2018, at 20:52, Ivan Kudryavtsev <kudryavtsev_ia@xxxxxxxxx>
> >>> wrote:
> >>>>
> >>>> Hello, dev team. Now I try to put a PR and need to write marvin test
> for
> >>>> it. I did it in the past, so tried to recall how to run the ecosystem.
> >>>>
> >>>> I started from the document:
> >>>>
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python
> >>>>
> >>>> To run simulator I have built docker image for 4.11:
> >>>>
> >>>> docker build -t simulator:4.11 -f tools/docker/Dockerfile .
> >>>>
> >>>> It has built correctly.
> >>>>
> >>>> Next, I run the simulator:
> >>>> docker run -it --rm -p 8080:8080 -p 8096:8096 simulator:411
> >>>>
> >>>> It started well, I can login.
> >>>>
> >>>> Next, I deployed DC as specified in document above:
> >>>> python tools/marvin/marvin/deployDataCenter.py
> >>>> <
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/deployDataCenter.py
> >>>>
> >>>> \
> >>>>   -i setup/dev/advanced.cfg
> >>>> <https://cwiki.apache.org/confluence/display/CLOUDSTACK/advanced.cfg>
> >>>>
> >>>> Next, I run the suite:
> >>>> $ export MARVIN_CONFIG=setup/dev/advanced.cfg
> >>>> <https://cwiki.apache.org/confluence/display/CLOUDSTACK/advanced.cfg>
> >>>> $ export TEST_SUITE=test/integration/smoke
> >>>> $ export ZONE_NAME=Sandbox-simulator
> >>>> $ nosetests-2.7 \
> >>>> --with-marvin \
> >>>> --marvin-config=${MARVIN_CONFIG} \
> >>>> -w ${TEST_SUITE} \
> >>>> --with-xunit \
> >>>> --xunit-file=/tmp/bvt_selfservice_cases.xml
> >>>> <
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/bvt_selfservice_cases.xml
> >>>>
> >>>> \
> >>>> --zone=${ZONE_NAME} \
> >>>> --hypervisor=simulator \
> >>>> -a tags=advanced,required_hardware=false
> >>>>
> >>>> What I see is that some tests fail, especially those, which connected
> >>> with
> >>>> OOB management.
> >>>> I tried to do the same for basic zone with basic.cfg and also OOB
> tests
> >>>> fail and some other as well.
> >>>>
> >>>> Is it OK or I do something wrong?
> >>>>
> >>>> Thanks a lot.
> >>>>
> >>>>
> >>>> --
> >>>> With best regards, Ivan Kudryavtsev
> >>>> Bitworks LLC
> >>>> Cell RU: +7-923-414-1515
> >>>> Cell USA: +1-201-257-1512
> >>>> WWW: http://bitworks.software/ <http://bw-sw.com/>
> >>>
> >>>
> >>
> >> --
> >> With best regards, Ivan Kudryavtsev
> >> Bitworks LLC
> >> Cell RU: +7-923-414-1515
> >> Cell USA: +1-201-257-1512
> >> WWW: http://bitworks.software/ <http://bw-sw.com/>
> >>
> >>
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks LLC
> > Cell RU: +7-923-414-1515
> > Cell USA: +1-201-257-1512
> > WWW: http://bitworks.software/ <http://bw-sw.com/>
>
>

-- 
With best regards, Ivan Kudryavtsev
Bitworks LLC
Cell RU: +7-923-414-1515
Cell USA: +1-201-257-1512
WWW: http://bitworks.software/ <http://bw-sw.com/>