Project

Profile

Help

Issue #3739

closed

Migrations failure during vagrant up

Added by ttereshc almost 6 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Dev Environment
Sprint:
Sprint 39
Quarter:

Description

To reproduce: remove all migrations from FS for core and plugins, do a fresh vagrant up.

Very long one-line error below but important last parts I put on new lines.
Notice there is no pulp_app migration in the list of running migrations. I believe that's the source of the problem because reset_db is done for every plugin.
This role runs reset_db for every enabled plugin.

TASK [dev : Create and run migrations for {{ pulp_app }}] **********************

TASK [django_db : Make migrations for pulp_app] ********************************
changed: [pulp3_dev]

TASK [django_db : Run all migrations] ******************************************
changed: [pulp3_dev] => (item=reset_db --noinput)
changed: [pulp3_dev] => (item=migrate auth --noinput)
changed: [pulp3_dev] => (item=migrate --noinput)
changed: [pulp3_dev] => (item=reset-admin-password --password admin)

TASK [plugin : Get information about /home/vagrant/devel/pulp_file] ************
ok: [pulp3_dev]

TASK [plugin : debug] **********************************************************
skipping: [pulp3_dev]

TASK [plugin : Install plugin packages into core Pulp Python environment] ******
changed: [pulp3_dev]

TASK [plugin : Run migrations for pulp_file] ***********************************

TASK [django_db : Make migrations for pulp_file] *******************************
changed: [pulp3_dev]

TASK [django_db : Run all migrations] ******************************************
changed: [pulp3_dev] => (item=reset_db --noinput)
changed: [pulp3_dev] => (item=migrate auth --noinput)
failed: [pulp3_dev] (item=migrate --noinput) => {"changed": true, "cmd": ["/home/vagrant/.virtualenvs/pulp/bin/pulp-manager", "migrate", "--noinput"], "delta": "0:00:01.225138", "end": "2018-06-05 16:57:59.954349", "item": "migrate --noinput", "msg": "non-zero return code", "rc": 1, "start": "2018-06-05 16:57:58.729211", "stderr": "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.\n  \"\"\")\nTraceback (most recent call last):\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute\n    return self.cursor.execute(sql, params)\npsycopg2.ProgrammingError: relation \"pulp_app_user\" does not exist\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/home/vagrant/.virtualenvs/pulp/bin/pulp-manager\", line 11, in <module>\n    sys.exit(pulp_manager_entry_point())\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/pulpcore/app/entry_points.py\", line 9, in pulp_manager_entry_point\n    execute_from_command_line(sys.argv)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", line 371, in execute_from_command_line\n    utility.execute()\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", line 365, in execute\n    self.fetch_command(subcommand).run_from_argv(self.argv)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/base.py\", line 288, in run_from_argv\n    self.execute(*args, **cmd_options)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/base.py\", line 335, in execute\n    output = self.handle(*args, **options)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/commands/migrate.py\", line 200, in handle\n    fake_initial=fake_initial,\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 117, in migrate\n    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 147, in _migrate_all_forwards\n    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 244, in apply_migration\n    state = migration.apply(state, schema_editor)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/base/schema.py\", line 106, in __exit__\n    self.execute(sql)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/base/schema.py\", line 133, in execute\n    cursor.execute(sql, params)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 100, in execute\n    return super().execute(sql, params)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 68, in execute\n    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 77, in _execute_with_wrappers\n    return executor(sql, params, many, context)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute\n    return self.cursor.execute(sql, params)\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/utils.py\", line 89, in __exit__\n    raise dj_exc_value.with_traceback(traceback) from exc_value\n  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute\n    return self.cursor.execute(sql, params)\ndjango.db.utils.ProgrammingError: relation \"pulp_app_user\" does not exist", "stderr_lines": ["/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.", "  \"\"\")", "Traceback (most recent call last):", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute", "    return self.cursor.execute(sql, params)", "psycopg2.ProgrammingError: relation \"pulp_app_user\" does not exist", "", "", "The above exception was the direct cause of the following exception:", "", "Traceback (most recent call last):", "  File \"/home/vagrant/.virtualenvs/pulp/bin/pulp-manager\", line 11, in <module>", "    sys.exit(pulp_manager_entry_point())", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/pulpcore/app/entry_points.py\", line 9, in pulp_manager_entry_point", "    execute_from_command_line(sys.argv)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", line 371, in execute_from_command_line", "    utility.execute()", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", line 365, in execute", "    self.fetch_command(subcommand).run_from_argv(self.argv)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/base.py\", line 288, in run_from_argv", "    self.execute(*args, **cmd_options)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/base.py\", line 335, in execute", "    output = self.handle(*args, **options)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/core/management/commands/migrate.py\", line 200, in handle", "    fake_initial=fake_initial,", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 117, in migrate", "    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 147, in _migrate_all_forwards", "    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/migrations/executor.py\", line 244, in apply_migration", "    state = migration.apply(state, schema_editor)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/base/schema.py\", line 106, in __exit__", "    self.execute(sql)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/base/schema.py\", line 133, in execute", "    cursor.execute(sql, params)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 100, in execute", "    return super().execute(sql, params)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 68, in execute", "    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 77, in _execute_with_wrappers", "    return executor(sql, params, many, context)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute", "    return self.cursor.execute(sql, params)", "  File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/utils.py\", line 89, in __exit__", "    raise dj_exc_value.with_traceback(traceback) from exc_value", " 
File \"/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute", "    return self.cursor.execute(sql, params)", "django.db.utils.ProgrammingError: relation \"pulp_app_user\" does not exist"], "stdout": "Operations to perform:\n  Apply all migrations: admin, auth, contenttypes, pulp_file, sessions\nRunning migrations:\n  
Applying admin.0001_initial...", "stdout_lines": ["Operations to perform:", "  Apply all migrations: admin, auth, contenttypes, pulp_file, sessions", "Running migrations:", "  Applying admin.0001_initial..."]}

I don't think it matters but , FWIW, I have 4 plugins enabled:

diff --git a/ansible/pulp-from-source.yml b/ansible/pulp-from-source.yml
index 4eef989..fa22953 100644
--- a/ansible/pulp-from-source.yml
+++ b/ansible/pulp-from-source.yml
@@ -21,7 +21,9 @@
     - redis
     - dev
     - {role: plugin, plugin_name: 'pulp_file', app_label: 'pulp_file'}
-    # - {role: plugin, plugin_name: 'pulp_python', app_label: 'pulp_python'}
+    - {role: plugin, plugin_name: 'pulp_python', app_label: 'pulp_python'}
+    - {role: plugin, plugin_name: 'pulp_rpm', app_label: 'pulp_rpm'}
+    - {role: plugin, plugin_name: 'pulp_ansible', app_label: 'pulp_ansible'}
     - systemd
     - role: dev_tools
       when: ansible_distribution == 'Fedora'

Removing the reset_db step helps but I don't think it's a solution. We'd better reset but one time at the beginning. Another question is why/how pulp_app migrations disappear.


Related issues

Related to Pulp - Issue #3742: Developer ansible installer migrations are not idempotent when used after systemdCLOSED - CURRENTRELEASEdalleyActions
Actions #1

Updated by ttereshc almost 6 years ago

  • Description updated (diff)
Actions #2

Updated by CodeHeeler almost 6 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 38
Actions #3

Updated by CodeHeeler almost 6 years ago

  • Related to Issue #3742: Developer ansible installer migrations are not idempotent when used after systemd added
Actions #4

Updated by ttereshc almost 6 years ago

  • Tags Pulp 3 added
Actions #5

Updated by dalley almost 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
Actions #6

Updated by rchan almost 6 years ago

  • Sprint changed from Sprint 38 to Sprint 39
Actions #7

Updated by dalley almost 6 years ago

  • Status changed from ASSIGNED to POST

Added by dalley almost 6 years ago

Revision 4aa6fd29 | View on GitHub

Only do "reset_db" once, instead of once per plugin

closes #3739 https://pulp.plan.io/issues/3739

Actions #8

Updated by dalley almost 6 years ago

  • Status changed from POST to MODIFIED
Actions #9

Updated by daviddavis about 5 years ago

  • Sprint/Milestone set to 3.0.0
Actions #10

Updated by bmbouter almost 5 years ago

  • Tags deleted (Pulp 3)
Actions #11

Updated by bmbouter over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF