Project

Profile

Help

Issue #3739

Migrations failure during vagrant up

Added by ttereshc over 1 year ago. Updated 6 months ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Severity:
2. Medium
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Dev Environment
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 39

Description

To reproduce: remove all migrations from FS for core and plugins, do a fresh vagrant up.

Very long one-line error below but important last parts I put on new lines.
Notice there is no pulp_app migration in the list of running migrations. I believe that's the source of the problem because reset_db is done for every plugin.
This role runs reset_db for every enabled plugin.

TASK [dev : Create and run migrations for {{ pulp_app }}] **********************

TASK [django_db : Make migrations for pulp_app] ********************************
changed: [pulp3_dev]

TASK [django_db : Run all migrations] ******************************************
changed: [pulp3_dev] => (item=reset_db --noinput)
changed: [pulp3_dev] => (item=migrate auth --noinput)
changed: [pulp3_dev] => (item=migrate --noinput)
changed: [pulp3_dev] => (item=reset-admin-password --password admin)

TASK [plugin : Get information about /home/vagrant/devel/pulp_file] ************
ok: [pulp3_dev]

TASK [plugin : debug] **********************************************************
skipping: [pulp3_dev]

TASK [plugin : Install plugin packages into core Pulp Python environment] ******
changed: [pulp3_dev]

TASK [plugin : Run migrations for pulp_file] ***********************************

TASK [django_db : Make migrations for pulp_file] *******************************
changed: [pulp3_dev]

TASK [django_db : Run all migrations] ******************************************
changed: [pulp3_dev] => (item=reset_db --noinput)
changed: [pulp3_dev] => (item=migrate auth --noinput)
failed: [pulp3_dev] (item=migrate --noinput) => {"changed": true, "cmd": ["/home/vagrant/.virtualenvs/pulp/bin/pulp-manager", "migrate", "--noinput"], "delta": "0:00:01.225138", "end": "2018-06-05 16:57:59.954349", "item": "migrate --noinput", "msg": "non-zero return code", "rc": 1, "start": "2018-06-05 16:57:58.729211", "stderr": "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.\n  \"\"\")\nTraceback (most recent call last):\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute\n    return self.cursor.execute(sql, params)\npsycopg2.ProgrammingError: relation \"pulp_app_user\" does not exist\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/home/vagrant/.virtualenvs/pulp/bin/pulp-manager\", line 11, in <module>\n    sys.exit(pulp_manager_entry_point())\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/pulpcore/app/entry_points.py\", line 9, in pulp_manager_entry_point\n    execute_from_command_line(sys.argv)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", line 371, in execute_from_command_line\n    utility.execute()\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", line 365, in execute\n    self.fetch_command(subcommand).run_from_argv(self.argv)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/base.py\", line 288, in run_from_argv\n    self.execute(*args, **cmd_options)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/base.py\", line 335, in execute\n    output = self.handle(*args, **options)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/commands/migrate.py\", line 200, in handle\n    fake_initial=fake_initial,\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 117, in migrate\n    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 147, in _migrate_all_forwards\n    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 244, in apply_migration\n    state = migration.apply(state, schema_editor)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/base/schema.py\", line 106, in __exit__\n    self.execute(sql)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/base/schema.py\", line 133, in execute\n    cursor.execute(sql, params)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 100, in execute\n    return super().execute(sql, params)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 68, in execute\n    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 77, in _execute_with_wrappers\n    return executor(sql, params, many, context)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute\n    return self.cursor.execute(sql, params)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/utils.py\", line 89, in __exit__\n    raise dj_exc_value.with_traceback(traceback) from exc_value\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute\n    return self.cursor.execute(sql, params)\ndjango.db.utils.ProgrammingError: relation \"pulp_app_user\" does not exist", "stderr_lines": ["/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.", "  \"\"\")", "Traceback (most recent call last):", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute", "    return self.cursor.execute(sql, params)", "psycopg2.ProgrammingError: relation \"pulp_app_user\" does not exist", "", "", "The above exception was the direct cause of the following exception:", "", "Traceback (most recent call last):", "  File \"/home/vagrant/.virtualenvs/pulp/bin/pulp-manager\", line 11, in <module>", "    sys.exit(pulp_manager_entry_point())", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/pulpcore/app/entry_points.py\", line 9, in pulp_manager_entry_point", "    execute_from_command_line(sys.argv)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", line 371, in execute_from_command_line", "    utility.execute()", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", line 365, in execute", "    self.fetch_command(subcommand).run_from_argv(self.argv)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/base.py\", line 288, in run_from_argv", "    self.execute(*args, **cmd_options)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/base.py\", line 335, in execute", "    output = self.handle(*args, **options)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/commands/migrate.py\", line 200, in handle", "    fake_initial=fake_initial,", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 117, in migrate", "    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 147, in _migrate_all_forwards", "    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 244, in apply_migration", "    state = migration.apply(state, schema_editor)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/base/schema.py\", line 106, in __exit__", "    self.execute(sql)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/base/schema.py\", line 133, in execute", "    cursor.execute(sql, params)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 100, in execute", "    return super().execute(sql, params)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 68, in execute", "    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 77, in _execute_with_wrappers", "    return executor(sql, params, many, context)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute", "    return self.cursor.execute(sql, params)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/utils.py\", line 89, in __exit__", "    raise dj_exc_value.with_traceback(traceback) from exc_value", " 
File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute", "    return self.cursor.execute(sql, params)", "django.db.utils.ProgrammingError: relation \"pulp_app_user\" does not exist"], "stdout": "Operations to perform:\n  Apply all migrations: admin, auth, contenttypes, pulp_file, sessions\nRunning migrations:\n  
Applying admin.0001_initial...", "stdout_lines": ["Operations to perform:", "  Apply all migrations: admin, auth, contenttypes, pulp_file, sessions", "Running migrations:", "  Applying admin.0001_initial..."]}

I don't think it matters but , FWIW, I have 4 plugins enabled:

diff --git a/ansible/pulp-from-source.yml b/ansible/pulp-from-source.yml
index 4eef989..fa22953 100644
--- a/ansible/pulp-from-source.yml
+++ b/ansible/pulp-from-source.yml
@@ -21,7 +21,9 @@
     - redis
     - dev
     - {role: plugin, plugin_name: 'pulp_file', app_label: 'pulp_file'}
-    # - {role: plugin, plugin_name: 'pulp_python', app_label: 'pulp_python'}
+    - {role: plugin, plugin_name: 'pulp_python', app_label: 'pulp_python'}
+    - {role: plugin, plugin_name: 'pulp_rpm', app_label: 'pulp_rpm'}
+    - {role: plugin, plugin_name: 'pulp_ansible', app_label: 'pulp_ansible'}
     - systemd
     - role: dev_tools
       when: ansible_distribution == 'Fedora'

Removing the reset_db step helps but I don't think it's a solution. We'd better reset but one time at the beginning. Another question is why/how pulp_app migrations disappear.


Related issues

Related to Pulp - Issue #3742: Developer ansible installer migrations are not idempotent when used after systemd MODIFIED Actions

Associated revisions

Revision 4aa6fd29 View on GitHub
Added by dalley over 1 year ago

Only do "reset_db" once, instead of once per plugin

closes #3739
https://pulp.plan.io/issues/3739

History

#1 Updated by ttereshc over 1 year ago

  • Description updated (diff)

#2 Updated by CodeHeeler over 1 year ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 38

#3 Updated by CodeHeeler over 1 year ago

  • Related to Issue #3742: Developer ansible installer migrations are not idempotent when used after systemd added

#4 Updated by ttereshc over 1 year ago

  • Tags Pulp 3 added

#5 Updated by dalley over 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

#6 Updated by rchan over 1 year ago

  • Sprint changed from Sprint 38 to Sprint 39

#7 Updated by dalley over 1 year ago

  • Status changed from ASSIGNED to POST

#8 Updated by dalley over 1 year ago

  • Status changed from POST to MODIFIED

#9 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#10 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF