https://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-02-18T13:13:02ZPulpPulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=678022021-02-18T13:13:02Zwibbit
<ul></ul><p>IRC Conversation with Mr Gainey</p>
<pre><code>ggainey
mornin wibbit - appears you were having fun last night
hm
the uploadchunks error - I assume that the core_uploadchunk table has rows in it in your db, yeah?
so yeah, the first pastebin will be resolved when 0052_tasking_logging_cid.py is applied, but the presence of rows in uploadchunk is failing the migration
ggainey
wibbit: I think Red Hat QE should start paying you a stipend
ttereshc: ping - are you around? can you read ^^ and tell me if my reasoning is correct regarding wibbit's migration issue?
ggainey
oh argh, I forgot DevConf starts today
mcorr
:)
wibbit
Would it be useful to send the previous versions I had, not just PulpCore
ggainey
wibbit: can you look into the db you're trying to upgrade, and tell me if core_uploadchunks has rows in it?
wibbit
ggainey: regarding rows in core_uploadchunk, yes, I've got 1272 rows in that table
ggainey
yeah, that's the problem - the migration doesn't work, because it wants to add the file column, which has no default, which works fine *if the table is empty*
this is a bug that we need to fix
ugh, the 'obvious' workaround is "truncate the table" and then apply the migration
wibbit
I can probably leave my system as is and await the patch, though I'm guessing this is easy enough to test, if this is infact the issue
I take it, this table not being empty isn't unusual, just not some thing that was caught with the test
s
ggainey
well, I haven't been in this workflow, so I'm guessing - but uploadchunk rows get deleted once the whole file has been uploaded. I think.
so this table usually is empty. I think.
wibbit
Interesting, so it could be related to failed uploads, as I don't wipe this DB between my integration testing.
ggainey
yeah, that's what it feels like to me
wibbit
I use the pulp api, to delete things like publications etc, so that I can re-create them.
In fact, looking at these dates they are back in october
ggainey
so my suggestion is thus - can you, please, open an issue with this discussion in it, because we do need to handle this case. Then, for your own sanity, you can truncate that table in your DB, and go on your merry upgrade-way
</code></pre> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=678692021-02-19T15:39:30Zfao89
<ul><li><strong>Triaged</strong> changed from <i>No</i> to <i>Yes</i></li><li><strong>Sprint</strong> set to <i>Sprint 91</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=683382021-03-05T20:09:44Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 91</i> to <i>Sprint 92</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=689082021-03-19T20:25:42Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 92</i> to <i>Sprint 93</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=694092021-04-06T21:53:06Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 93</i> to <i>Sprint 94</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=698232021-04-16T21:13:13Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 94</i> to <i>Sprint 95</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=702652021-04-30T16:13:31Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 95</i> to <i>Sprint 96</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=707852021-05-14T15:34:00Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 96</i> to <i>Sprint 97</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=715822021-06-02T16:02:23Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 97</i> to <i>Sprint 98</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=720582021-06-17T20:11:24Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 98</i> to <i>Sprint 99</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=725932021-06-30T20:36:44Zlmjachky
<ul></ul><p>Based on the attached discussion, the issue could be really caused by uncommitted uploads (<a href="https://docs.pulpproject.org/pulpcore/changes.html#id230" class="external">https://docs.pulpproject.org/pulpcore/changes.html#id230</a>). Since we encouraged users to purge all uncommitted uploaded files, I believe the problem is not on our side. Still, further investigation is required.</p> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=727242021-07-02T19:23:14Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 99</i> to <i>Sprint 100</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=730112021-07-10T03:34:57Zdalleydalley@redhat.com
<ul></ul><p>If uncommitted uploads can actually cause upgrade failures then that warning is definitely insufficient, unless it was called out explicitly in the announcement email as well. Manually adding some text to the changelog to make it more prominent wouldn't hurt either.</p>
<p>Just a note for the future, probably not worth worrying too much about now.</p> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=732892021-07-15T21:08:15Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 100</i> to <i>Sprint 101</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=738652021-07-28T14:59:08Zdalleydalley@redhat.com
<ul><li><strong>Tags</strong> <i>Documentation</i> added</li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=740822021-08-02T17:44:19Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Sprint</strong> changed from <i>Sprint 101</i> to <i>Sprint 102</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=744592021-08-12T15:23:42Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 102</i> to <i>Sprint 103</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=745302021-08-17T13:40:06Zttereshcttereshc@redhat.com
<ul><li><strong>Quarter</strong> set to <i>Q3-2021</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=749742021-08-27T15:08:01Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 103</i> to <i>Sprint 104</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=754072021-09-09T22:28:10Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 104</i> to <i>Sprint 105</i></li></ul> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=754522021-09-09T23:27:24Zlmjachky
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>CLOSED - WONTFIX</i></li></ul><p>I am closing this because the issue is relevant to uncommitted uploads and we cannot fix it.</p>
<p>Lessons learned:</p>
<ul>
<li>send announcements and warnings about potential errors that may arise during the migrations and do not leave the encouragements in changelogs only...</li>
</ul>
<pre><code>Removals
* The local file system directory used for uploaded chunks is specified by the setting CHUNKED_UPLOAD_DIR. Users are encouraged to remove all uncommitted uploaded files before applying this change. #4498
</code></pre> Pulp - Issue #8277: Pulp upgrade fails - core.0049_add_file_field_to_uploadchunkhttps://pulp.plan.io/issues/8277?journal_id=754602021-09-10T10:51:56Zlmjachky
<ul></ul><p>The commit which initially introduced the problem is right here: <a href="https://github.com/pulp/pulpcore/commit/1b6c7360c30cc7d9e2d3b9fc5062ac1f7d69c2de" class="external">https://github.com/pulp/pulpcore/commit/1b6c7360c30cc7d9e2d3b9fc5062ac1f7d69c2de</a>. Before this change, we were appending chunks to a single file. Right now, we have one file per uploaded chunk.</p>
<p>Note that by deleting uncommitted uploads, I mean deleting DB records. Deleting the DB records should eventually result in deleting the referenced files (uploads) on the system storage.</p>
<p>This step (deleting uncommitted uploads) was necessary to accomplish due to the fact that the new chunked uploading system could not properly determine how should a single file be split into multiple files representing each uploaded chunk during the migration.</p>