git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: RFC. Marvin tests fail


you could run it with a debugger in PyCharm for example and figure out if there’s anything wrong with the code. Please note that these tests are successful with KVM, VMware and Xen.

The logs you’ve shared are just results, there’s a failed_plus_exceptions file in each test file log, where you could find more detailed info and stack trace. 

Boris.


boris.stoyanov@xxxxxxxxxxxxx 
www.shapeblue.com
Amadeus House, Floral Street, London  WC2E 9DPUK
@shapeblue
  
 

> On 8 Nov 2018, at 2:37, Ivan Kudryavtsev <kudryavtsev_ia@xxxxxxxxx> wrote:
> 
> Looks like the reason for the failure is in python code, not tests. Someone
> broke tests.
> 
> ср, 7 нояб. 2018 г. в 19:29, Ivan Kudryavtsev <kudryavtsev_ia@xxxxxxxxx>:
> 
>> Well, when I run outside of docker. I see another behavior:
>> Those OOB tests work well, but I meet errors in other pieces of code:
>> 
>> *ivan@notebook:/tmp/MarvinLogs/Nov_07_2018_18_53_35_8SWDQD$ grep -P
>> 'EXCEPTION|FAIL' results.txt *
>> Tests ha enable/disable feature at cluster and zone level ... ===
>> TestName: test_ha_configure_enabledisable_across_clusterzones | Status :
>> EXCEPTION ===
>> Tests ha resource ownership expiry across multi-mgmt server ... ===
>> TestName: test_ha_multiple_mgmt_server_ownership | Status : EXCEPTION ===
>> Tests ha FSM transitions for valid healthy host ... === TestName:
>> test_ha_verify_fsm_available | Status : EXCEPTION ===
>> Tests ha FSM transitions leading to degraded state ... === TestName:
>> test_ha_verify_fsm_degraded | Status : EXCEPTION ===
>> Tests ha FSM transitions for failures leading to fenced state ... ===
>> TestName: test_ha_verify_fsm_fenced | Status : EXCEPTION ===
>> Tests ha FSM transitions leading to recovering ... === TestName:
>> test_ha_verify_fsm_recovering | Status : EXCEPTION ===
>> === TestName: test_list_zones_metrics | Status : EXCEPTION ===
>> Tests out-of-band management ownership expiry across multi-mgmt server ...
>> === TestName: test_oobm_multiple_mgmt_server_ownership | Status : FAILED ===
>> FAIL
>> === TestName: test_ha_configure_enabledisable_across_clusterzones | Status
>> : EXCEPTION ===
>> === TestName: test_ha_multiple_mgmt_server_ownership | Status : EXCEPTION
>> ===
>> === TestName: test_ha_verify_fsm_available | Status : EXCEPTION ===
>> === TestName: test_ha_verify_fsm_degraded | Status : EXCEPTION ===
>> === TestName: test_ha_verify_fsm_fenced | Status : EXCEPTION ===
>> === TestName: test_ha_verify_fsm_recovering | Status : EXCEPTION ===
>> FAIL: Tests out-of-band management ownership expiry across multi-mgmt
>> server
>> === TestName: test_oobm_multiple_mgmt_server_ownership | Status : FAILED
>> ===
>> FAILED (SKIP=13, errors=6, failures=1)
>> 
>> I pasted errors to Pastebin: https://pastebin.com/tXw6mRk7
>> 
>> 
>> 
>> 
>> ср, 7 нояб. 2018 г. в 14:05, Boris Stoyanov <boris.stoyanov@xxxxxxxxxxxxx
>>> :
>> 
>>> Hi Ivan,
>>> I guess you’re referring to Out of Band management? I think there was a
>>> simulator provider for those tests and they are reported as passing with
>>> latest KVM test run.
>>> 
>>> Can you share any logs/exceptions? I don’t see anything wrong with your
>>> setup on a first glance btw.
>>> 
>>> Bobby.
>>> 
>>> 
>>> boris.stoyanov@xxxxxxxxxxxxx
>>> www.shapeblue.com
>>> Amadeus House, Floral Street, London  WC2E 9DPUK
>>> @shapeblue
>>> 
>>> 
>>> 
>>>> On 7 Nov 2018, at 20:52, Ivan Kudryavtsev <kudryavtsev_ia@xxxxxxxxx>
>>> wrote:
>>>> 
>>>> Hello, dev team. Now I try to put a PR and need to write marvin test for
>>>> it. I did it in the past, so tried to recall how to run the ecosystem.
>>>> 
>>>> I started from the document:
>>>> 
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python
>>>> 
>>>> To run simulator I have built docker image for 4.11:
>>>> 
>>>> docker build -t simulator:4.11 -f tools/docker/Dockerfile .
>>>> 
>>>> It has built correctly.
>>>> 
>>>> Next, I run the simulator:
>>>> docker run -it --rm -p 8080:8080 -p 8096:8096 simulator:411
>>>> 
>>>> It started well, I can login.
>>>> 
>>>> Next, I deployed DC as specified in document above:
>>>> python tools/marvin/marvin/deployDataCenter.py
>>>> <
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/deployDataCenter.py
>>>> 
>>>> \
>>>>   -i setup/dev/advanced.cfg
>>>> <https://cwiki.apache.org/confluence/display/CLOUDSTACK/advanced.cfg>
>>>> 
>>>> Next, I run the suite:
>>>> $ export MARVIN_CONFIG=setup/dev/advanced.cfg
>>>> <https://cwiki.apache.org/confluence/display/CLOUDSTACK/advanced.cfg>
>>>> $ export TEST_SUITE=test/integration/smoke
>>>> $ export ZONE_NAME=Sandbox-simulator
>>>> $ nosetests-2.7 \
>>>> --with-marvin \
>>>> --marvin-config=${MARVIN_CONFIG} \
>>>> -w ${TEST_SUITE} \
>>>> --with-xunit \
>>>> --xunit-file=/tmp/bvt_selfservice_cases.xml
>>>> <
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/bvt_selfservice_cases.xml
>>>> 
>>>> \
>>>> --zone=${ZONE_NAME} \
>>>> --hypervisor=simulator \
>>>> -a tags=advanced,required_hardware=false
>>>> 
>>>> What I see is that some tests fail, especially those, which connected
>>> with
>>>> OOB management.
>>>> I tried to do the same for basic zone with basic.cfg and also OOB tests
>>>> fail and some other as well.
>>>> 
>>>> Is it OK or I do something wrong?
>>>> 
>>>> Thanks a lot.
>>>> 
>>>> 
>>>> --
>>>> With best regards, Ivan Kudryavtsev
>>>> Bitworks LLC
>>>> Cell RU: +7-923-414-1515
>>>> Cell USA: +1-201-257-1512
>>>> WWW: http://bitworks.software/ <http://bw-sw.com/>
>>> 
>>> 
>> 
>> --
>> With best regards, Ivan Kudryavtsev
>> Bitworks LLC
>> Cell RU: +7-923-414-1515
>> Cell USA: +1-201-257-1512
>> WWW: http://bitworks.software/ <http://bw-sw.com/>
>> 
>> 
> 
> -- 
> With best regards, Ivan Kudryavtsev
> Bitworks LLC
> Cell RU: +7-923-414-1515
> Cell USA: +1-201-257-1512
> WWW: http://bitworks.software/ <http://bw-sw.com/>