Project

Profile

Help

Test #4385

Investigate rpm.api_v2.test_updateinfo.OpenSuseErrataTestCase.test_01_create_sync_publish after OpenStack move

Added by bherring 8 months ago. Updated 6 months ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Version:
Platform Release:
Tags:
Pulp 2
Sprint:

Description

Problem

In Jenkins, there are flakey failures with `rpm.api_v2.test_updateinfo.OpenSuseErrataTestCase.test_01_create_sync_publish`.

Resolve issues.

Failure Output 1

Link: https://pulp-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Pulp2.18/job/pulp-2.18-dev-rhel7/24/testReport/rpm.api_v2.test_updateinfo/OpenSuseErrataTestCase/test_01_create_sync_publish_2/

Regression
rpm.api_v2.test_updateinfo.OpenSuseErrataTestCase.test_01_create_sync_publish (from pytest)

Failing for the past 1 build (Since Unstable#24 )
Took 0 ms.
add description
Error Message
test teardown failure
Stacktrace
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:420: in delete
    return self.request('DELETE', url, **kwargs)
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:481: in request
    requests.request(method, **request_kwargs),
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:148: in json_handler
    _handle_202(client._cfg, response, client.pulp_host)  # pylint:disable=protected-access
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:83: in _handle_202
    tasks = tuple(poll_spawned_tasks(cfg, call_report, pulp_host))
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:507: in poll_spawned_tasks
    for final_task_state in poll_task(cfg, href, pulp_host):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cfg = PulpSmashConfig(pulp_auth=['admin', 'admin'], pulp_version='2.18.1', pulp_selinux_enabled=True, hosts=[PulpHost(hostna...t': {}, 'pulp cli': {}, 'pulp resource manager': {}, 'pulp workers': {}, 'shell': {'transport': 'ssh'}, 'squid': {}})])
href = '/pulp/api/v2/tasks/2100578d-d271-4066-8711-d9312577f211/'
pulp_host = PulpHost(hostname='host-172-16-46-64.openstacklocal', roles={'amqp broker': {'service': 'qpidd'}, 'api': {'scheme': 'h...eat': {}, 'pulp cli': {}, 'pulp resource manager': {}, 'pulp workers': {}, 'shell': {'transport': 'ssh'}, 'squid': {}})

    def poll_task(cfg, href, pulp_host=None):
        """Wait for a task and its children to complete. Yield response bodies.

        Poll the task at ``href``, waiting for the task to complete. When a
        response is received indicating that the task is complete, yield that
        response body and recursively poll each child task.

        :param cfg: A :class:`pulp_smash.config.PulpSmashConfig` object.
        :param href: The path to a task you'd like to monitor recursively.
        :param pulp_host: The host to poll. If ``None``, a host will automatically
            be selected by :class:`Client`.
        :returns: An generator yielding response bodies.
        :raises pulp_smash.exceptions.TaskTimedOutError: If a task takes too
            long to complete.
        """ 
        # 900 * 2s == 1800s == 30m
        # NOTE: The timeout counter is synchronous. We query Pulp, then count down,
        # then query pulp, then count down, etc. This is… dumb.
        poll_limit = 900
        poll_counter = 0
        json_client = Client(cfg, json_handler, pulp_host=pulp_host)
        while True:
            task = json_client.get(href)
            if cfg.pulp_version < Version('3'):
                task_end_states = _TASK_END_STATES
            else:
                task_end_states = _P3_TASK_END_STATES
            if task['state'] in task_end_states:
                # This task has completed. Yield its final state, then recursively
                # iterate through children and yield their final states.
                yield task
                for spawned_task in task['spawned_tasks']:
                    for descendant_tsk in poll_task(cfg, spawned_task['_href'], pulp_host):
                        yield descendant_tsk
                break
            poll_counter += 1
            if poll_counter > poll_limit:
                raise exceptions.TaskTimedOutError(
>                   'Task {} is ongoing after {} polls.'.format(href, poll_limit)
                )
E               pulp_smash.exceptions.TaskTimedOutError: Task /pulp/api/v2/tasks/2100578d-d271-4066-8711-d9312577f211/ is ongoing after 900 polls.

../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:549: TaskTimedOutError

Failure Output 2

Link: https://pulp-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Pulp2.18/job/pulp-2.18-dev-rhel7/24/testReport/rpm.api_v2.test_updateinfo/OpenSuseErrataTestCase/test_01_create_sync_publish_2/

Regression
rpm.api_v2.test_updateinfo.OpenSuseErrataTestCase.test_01_create_sync_publish (from pytest)

Failing for the past 1 build (Since Unstable#24 )
Took 1 hr 14 min.
add description
Error Message
pulp_smash.exceptions.TaskTimedOutError: Task /pulp/api/v2/tasks/00b36a8e-7229-4517-9af6-98fd057a2af1/ is ongoing after 900 polls.
Stacktrace
self = <pulp_2_tests.tests.rpm.api_v2.test_updateinfo.OpenSuseErrataTestCase testMethod=test_01_create_sync_publish>

    def test_01_create_sync_publish(self):
        """Create, sync, and publish an openSUSE repository.""" 
        cfg = config.get_config()
        if not selectors.bug_is_fixed(3377, cfg.pulp_version):
            self.skipTest('https://pulp.plan.io/issues/3377')

        # Create, sync, and publish a repository.
        client = api.Client(cfg, api.json_handler)
        body = gen_repo()
        body['importer_config']['download_policy'] = 'on_demand'
        body['importer_config']['feed'] = OPENSUSE_FEED_URL
        body['distributors'] = [gen_distributor()]
        repo = client.post(REPOSITORY_PATH, body)
        self.addCleanup(client.delete, repo['_href'])
>       sync_repo(cfg, repo)

../../.virtualenvs/pulp-2-tests/lib64/python3.6/site-packages/pulp_2_tests/tests/rpm/api_v2/test_updateinfo.py:708: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/pulp2/utils.py:480: in sync_repo
    return api.Client(cfg).post(urljoin(repo['_href'], 'actions/sync/'))
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:443: in post
    return self.request('POST', url, **kwargs)
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:481: in request
    requests.request(method, **request_kwargs),
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:135: in safe_handler
    _handle_202(client._cfg, response, client.pulp_host)  # pylint:disable=protected-access
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:83: in _handle_202
    tasks = tuple(poll_spawned_tasks(cfg, call_report, pulp_host))
../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:507: in poll_spawned_tasks
    for final_task_state in poll_task(cfg, href, pulp_host):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cfg = PulpSmashConfig(pulp_auth=['admin', 'admin'], pulp_version='2.18.1', pulp_selinux_enabled=True, hosts=[PulpHost(hostna...t': {}, 'pulp cli': {}, 'pulp resource manager': {}, 'pulp workers': {}, 'shell': {'transport': 'ssh'}, 'squid': {}})])
href = '/pulp/api/v2/tasks/00b36a8e-7229-4517-9af6-98fd057a2af1/'
pulp_host = PulpHost(hostname='host-172-16-46-64.openstacklocal', roles={'amqp broker': {'service': 'qpidd'}, 'api': {'scheme': 'h...eat': {}, 'pulp cli': {}, 'pulp resource manager': {}, 'pulp workers': {}, 'shell': {'transport': 'ssh'}, 'squid': {}})

    def poll_task(cfg, href, pulp_host=None):
        """Wait for a task and its children to complete. Yield response bodies.

        Poll the task at ``href``, waiting for the task to complete. When a
        response is received indicating that the task is complete, yield that
        response body and recursively poll each child task.

        :param cfg: A :class:`pulp_smash.config.PulpSmashConfig` object.
        :param href: The path to a task you'd like to monitor recursively.
        :param pulp_host: The host to poll. If ``None``, a host will automatically
            be selected by :class:`Client`.
        :returns: An generator yielding response bodies.
        :raises pulp_smash.exceptions.TaskTimedOutError: If a task takes too
            long to complete.
        """ 
        # 900 * 2s == 1800s == 30m
        # NOTE: The timeout counter is synchronous. We query Pulp, then count down,
        # then query pulp, then count down, etc. This is… dumb.
        poll_limit = 900
        poll_counter = 0
        json_client = Client(cfg, json_handler, pulp_host=pulp_host)
        while True:
            task = json_client.get(href)
            if cfg.pulp_version < Version('3'):
                task_end_states = _TASK_END_STATES
            else:
                task_end_states = _P3_TASK_END_STATES
            if task['state'] in task_end_states:
                # This task has completed. Yield its final state, then recursively
                # iterate through children and yield their final states.
                yield task
                for spawned_task in task['spawned_tasks']:
                    for descendant_tsk in poll_task(cfg, spawned_task['_href'], pulp_host):
                        yield descendant_tsk
                break
            poll_counter += 1
            if poll_counter > poll_limit:
                raise exceptions.TaskTimedOutError(
>                   'Task {} is ongoing after {} polls.'.format(href, poll_limit)
                )
E               pulp_smash.exceptions.TaskTimedOutError: Task /pulp/api/v2/tasks/00b36a8e-7229-4517-9af6-98fd057a2af1/ is ongoing after 900 polls.

../../.virtualenvs/pulp-2-tests/lib/python3.6/site-packages/pulp_smash/api.py:549: TaskTimedOutError

History

#1 Updated by bmbouter 6 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF