Project

Profile

Help

Task #5878

closed

Write automated script that inspects all Remdine content and removes spam

Added by bmbouter almost 3 years ago. Updated about 2 years ago.

Status:
CLOSED - COMPLETE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

pulp.plan.io has a major spam problem. It's all over, hundreds of accounts, attachments, and posts.

Write an automated script that uses the redmine API to inspect all posts and determine if spam or not

If spam then:
1. lock the user account
2. remove all of their comments from all posts
3. delete all attachments uploaded by them
4. delete any issues they filed directly.

Actions #1

Updated by daviddavis almost 3 years ago

I agree 100% that the spam has become a major problem but I have a concern about this solution: most of these actions are irreversible. Imagine if the script were to accidentally identify one of your issues or comments as spam--all of your issues, comments, etc would be gone forever. Perhaps we could have some sort of human validation as a step (at least to begin, until we confirm that the script won't cause false positives)? Or maybe we could limit the script to people who aren't in a redmine User Group (ie new users)?

Actions #2

Updated by bmbouter almost 3 years ago

I agree, let's ensure that this script has a human with it when it runs to ensure that not one mistake is made. It should include human confirmation of spam and also confirmation of the things to delete pre-deletion. Overall the Redmine data is crucially important.

Actions #3

Updated by rchan over 2 years ago

We already have a pulpbot redmine user account that has the permissions to delete users as step #5 above. Or perhaps remove locked users who have been locked for a long time (another indication of spam or obsolete users.) Additionally, we can create a new user specifically for this clean up.

Actions #4

Updated by rchan over 2 years ago

  • Tags Backlog added
Actions #5

Updated by ttereshc over 2 years ago

The up-to-date REST API docs https://www.redmine.org/projects/redmine/wiki/Rest_api.

I asked Redmine support about the following actions supported through the API:

  • remove an issue (I believe it's supported)
  • remove a comment from an issue
  • remove an attachment from an issue
  • lock a user
  • we need to read/list different resources (users, issues, etc) individually and in bulk (I think it's supported)

Here is the reply:

I think everything except updating / removing journal notes should be possible right now. For the latter, there is an open issue on redmine.org. I'll see what we can do about this, maybe it's not too hard to implement. I can't promise you any implementation date right now unfortunately.
In order to remove an attachment, you would first have to get hold of it's ID (by getting the issue details including attachments), and then use the attachments API to delete it.

Actions #6

Updated by bmbouter over 2 years ago

I commented on the upstream API feature gap https://www.redmine.org/issues/10171#note-6

I'm in favor of us implementing spam cleanup with all of the API features we currently.

Actions #7

Updated by ipanova@redhat.com over 2 years ago

  • Tags deleted (Backlog)
Actions #8

Updated by lmjachky over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to lmjachky

Added by Lubos Mjachky over 2 years ago

Revision ebdc415d

Add a spam cleanup script

closes #5878 https://pulp.plan.io/issues/5878

Actions #9

Updated by lmjachky over 2 years ago

  • Status changed from ASSIGNED to POST
Actions #10

Updated by Anonymous over 2 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #11

Updated by ttereshc about 2 years ago

  • Status changed from MODIFIED to CLOSED - COMPLETE

Also available in: Atom PDF