Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OF-2954: New feature: Spam Reporting #2661

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

OF-2954: New feature: Spam Reporting #2661

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
9b82d8e
OF-2954: New feature: Spam Reporting
guusdk Jan 14, 2025
a4c4798
OF-2954: Make SpamReportProvider configurable
guusdk Jan 15, 2025
83fb015
OF-2954: Database install and update scripts
guusdk Jan 15, 2025
70fd67c
OF-2954: SpamReport should retain copy of detached element
guusdk Jan 15, 2025
729ec91
OF-2954: Include spam report in admin notification
guusdk Jan 15, 2025
91726c9
OF-2954: Refactoring
guusdk Jan 15, 2025
8ec5fa3
OF-2954: Add Spam Reporting admin console panel
guusdk Jan 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion distribution/src/database/openfire_db2.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -395,14 +395,24 @@ CREATE TABLE ofPubsubDefaultConf (
CONSTRAINT ofPubsubDefConf_pk PRIMARY KEY (serviceID, leaf)
);

CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the db2 has the TIMESTAMP type. In other places of the file used char for date, which it very strange

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(as above: we currently never use a TIMESTAMP field, which is certainly something I'd like to improve on. If we do improve on that, we should do it consistently, everywhere).

raw LONG VARCHAR NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

-- Finally, insert default table values
INSERT INTO ofID (idType, id) VALUES (18, 1);
INSERT INTO ofID (idType, id) VALUES (19, 1);
INSERT INTO ofID (idType, id) VALUES (23, 1);
INSERT INTO ofID (idType, id) VALUES (26, 2);
INSERT INTO ofID (idType, id) VALUES (27, 1);

INSERT INTO ofVersion (name, version) VALUES ('openfire', 36);
INSERT INTO ofVersion (name, version) VALUES ('openfire', 37);

-- Entry for admin user
INSERT INTO ofUser (username, plainPassword, name, email, creationDate, modificationDate)
Expand Down
12 changes: 11 additions & 1 deletion distribution/src/database/openfire_hsqldb.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -381,6 +381,16 @@ CREATE TABLE ofPubsubDefaultConf (
CONSTRAINT ofPubsubDefaultConf_pk PRIMARY KEY (serviceID, leaf)
);

CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
raw LONGVARCHAR NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

// Finally, insert default table values.

INSERT INTO ofID (idType, id) VALUES (18, 1);
Expand All @@ -389,7 +399,7 @@ INSERT INTO ofID (idType, id) VALUES (23, 1);
INSERT INTO ofID (idType, id) VALUES (26, 2);
INSERT INTO ofID (idType, id) VALUES (27, 1);

INSERT INTO ofVersion (name, version) VALUES ('openfire', 36);
INSERT INTO ofVersion (name, version) VALUES ('openfire', 37);

// Entry for admin user
INSERT INTO ofUser (username, plainPassword, name, email, creationDate, modificationDate)
Expand Down
12 changes: 11 additions & 1 deletion distribution/src/database/openfire_mysql.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -371,6 +371,16 @@ CREATE TABLE ofPubsubDefaultConf (
PRIMARY KEY (serviceID, leaf)
);

CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may use timestamp

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea, but not for now: We've had trouble before finding a consistent way to represent a timestamp in all of the databases that we support, which is why we use a number instead. I'm not sure if this is still as impossible as it was in 2004, by the way, but I'd like Openfire to be consistent.

If we do change number for timestamp (which would be a good thing), we should do it for all columns that currently use a number.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also see the remark at the top of https://download.igniterealtime.org/openfire/docs/latest/documentation/database-guide.html which describes this.

raw TEXT NOT NULL,
INDEX ofSpamReport_created_reporter_id (created, reporter),
INDEX ofSpamReport_created_reported_id (created, reported)
);

# Finally, insert default table values.

INSERT INTO ofID (idType, id) VALUES (18, 1);
Expand All @@ -379,7 +389,7 @@ INSERT INTO ofID (idType, id) VALUES (23, 1);
INSERT INTO ofID (idType, id) VALUES (26, 2);
INSERT INTO ofID (idType, id) VALUES (27, 1);

INSERT INTO ofVersion (name, version) VALUES ('openfire', 36);
INSERT INTO ofVersion (name, version) VALUES ('openfire', 37);

# Entry for admin user
INSERT INTO ofUser (username, plainPassword, name, email, creationDate, modificationDate)
Expand Down
12 changes: 11 additions & 1 deletion distribution/src/database/openfire_oracle.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,16 @@ CREATE TABLE ofPubsubDefaultConf (
CONSTRAINT ofPubsubDefaultConf_pk PRIMARY KEY (serviceID, leaf)
);

CREATE TABLE ofSpamReport (
reporter VARCHAR2(1024) NOT NULL,
reported VARCHAR2(1024) NOT NULL,
reason VARCHAR2(255) NOT NULL,
created INTEGER NOT NULL,
"raw" CLOB NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

-- Finally, insert default table values.

INSERT INTO ofID (idType, id) VALUES (18, 1);
Expand All @@ -387,7 +397,7 @@ INSERT INTO ofID (idType, id) VALUES (23, 1);
INSERT INTO ofID (idType, id) VALUES (26, 2);
INSERT INTO ofID (idType, id) VALUES (27, 1);

INSERT INTO ofVersion (name, version) VALUES ('openfire', 36);
INSERT INTO ofVersion (name, version) VALUES ('openfire', 37);

-- Entry for admin user
INSERT INTO ofUser (username, plainPassword, name, email, creationDate, modificationDate)
Expand Down
12 changes: 11 additions & 1 deletion distribution/src/database/openfire_postgresql.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -387,6 +387,16 @@ CREATE TABLE ofPubsubDefaultConf (
CONSTRAINT ofPubsubDefaultConf_pk PRIMARY KEY (serviceID, leaf)
);

CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
raw TEXT NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

-- Finally, insert default table values.

INSERT INTO ofID (idType, id) VALUES (18, 1);
Expand All @@ -395,7 +405,7 @@ INSERT INTO ofID (idType, id) VALUES (23, 1);
INSERT INTO ofID (idType, id) VALUES (26, 2);
INSERT INTO ofID (idType, id) VALUES (27, 1);

INSERT INTO ofVersion (name, version) VALUES ('openfire', 36);
INSERT INTO ofVersion (name, version) VALUES ('openfire', 37);

-- Entry for admin user
INSERT INTO ofUser (username, plainPassword, name, email, creationDate, modificationDate)
Expand Down
12 changes: 11 additions & 1 deletion distribution/src/database/openfire_sqlserver.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,16 @@ CREATE TABLE ofPubsubDefaultConf (
CONSTRAINT ofPubsubDefaultConf_pk PRIMARY KEY (serviceID, leaf)
);

CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
raw NVARCHAR(MAX) NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

/* Finally, insert default table values. */

INSERT INTO ofID (idType, id) VALUES (18, 1);
Expand All @@ -392,7 +402,7 @@ INSERT INTO ofID (idType, id) VALUES (23, 1);
INSERT INTO ofID (idType, id) VALUES (26, 2);
INSERT INTO ofID (idType, id) VALUES (27, 1);

INSERT INTO ofVersion (name, version) VALUES ('openfire', 36);
INSERT INTO ofVersion (name, version) VALUES ('openfire', 37);

/* Entry for admin user */
INSERT INTO ofUser (username, plainPassword, name, email, creationDate, modificationDate)
Expand Down
12 changes: 11 additions & 1 deletion distribution/src/database/openfire_sybase.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,16 @@ CREATE TABLE ofPubsubDefaultConf (
CONSTRAINT ofPubsubDefaultConf_pk PRIMARY KEY (serviceID, leaf)
)

CREATE TABLE ofSpamReport (
reporter NVARCHAR(1024) NOT NULL,
reported NVARCHAR(1024) NOT NULL,
reason NVARCHAR(255) NOT NULL,
created INTEGER NOT NULL,
"raw" LONG VARCHAR NOT NULL
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe rename the raw field?
It contains the "XML representation of the report" but maybe we just need only the reported message so that we can easily parse the spam and train spamd.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That column is not great, indeed. I'm in the middle of a refactoring of the database structure. I think this column will be dropped (and instead we'll have a table that can be JOINed with, to have zero-to-many reported stanzas. More on this later!

)
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter)
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported)

/* Finally, insert default table values. */

INSERT INTO ofID (idType, id) VALUES (18, 1)
Expand All @@ -392,7 +402,7 @@ INSERT INTO ofID (idType, id) VALUES (23, 1)
INSERT INTO ofID (idType, id) VALUES (26, 2)
INSERT INTO ofID (idType, id) VALUES (27, 1)

INSERT INTO ofVersion (name, version) VALUES ('openfire', 36)
INSERT INTO ofVersion (name, version) VALUES ('openfire', 37)

/* Entry for admin user */
INSERT INTO ofUser (username, plainPassword, name, email, creationDate, modificationDate)
Expand Down
11 changes: 11 additions & 0 deletions distribution/src/database/upgrade/37/openfire_db2.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
raw LONG VARCHAR NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

UPDATE ofVersion SET version = 37 WHERE name = 'openfire';
11 changes: 11 additions & 0 deletions distribution/src/database/upgrade/37/openfire_hsqldb.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
raw LONGVARCHAR NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

UPDATE ofVersion SET version = 37 WHERE name = 'openfire';
11 changes: 11 additions & 0 deletions distribution/src/database/upgrade/37/openfire_mysql.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
raw TEXT NOT NULL,
INDEX ofSpamReport_created_reporter_id (created, reporter),
INDEX ofSpamReport_created_reported_id (created, reported)
);

UPDATE ofVersion SET version = 37 WHERE name = 'openfire';
11 changes: 11 additions & 0 deletions distribution/src/database/upgrade/37/openfire_oracle.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
CREATE TABLE ofSpamReport (
reporter VARCHAR2(1024) NOT NULL,
reported VARCHAR2(1024) NOT NULL,
reason VARCHAR2(255) NOT NULL,
created INTEGER NOT NULL,
"raw" CLOB NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

UPDATE ofVersion SET version = 37 WHERE name = 'openfire';
11 changes: 11 additions & 0 deletions distribution/src/database/upgrade/37/openfire_postgresql.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
raw TEXT NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

UPDATE ofVersion SET version = 37 WHERE name = 'openfire';
11 changes: 11 additions & 0 deletions distribution/src/database/upgrade/37/openfire_sqlserver.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
CREATE TABLE ofSpamReport (
reporter VARCHAR(1024) NOT NULL,
reported VARCHAR(1024) NOT NULL,
reason VARCHAR(255) NOT NULL,
created BIGINT NOT NULL,
raw NVARCHAR(MAX) NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

UPDATE ofVersion SET version = 37 WHERE name = 'openfire';
11 changes: 11 additions & 0 deletions distribution/src/database/upgrade/37/openfire_sybase.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
CREATE TABLE ofSpamReport (
reporter NVARCHAR(1024) NOT NULL,
reported NVARCHAR(1024) NOT NULL,
reason NVARCHAR(255) NOT NULL,
created INTEGER NOT NULL,
"raw" LONG VARCHAR NOT NULL
);
CREATE INDEX ofSpamReport_created_reporter_id ON ofSpamReport (created, reporter);
CREATE INDEX ofSpamReport_created_reported_id ON ofSpamReport (created, reported);

UPDATE ofVersion SET version = 37 WHERE name = 'openfire';
52 changes: 50 additions & 2 deletions documentation/database-guide.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,10 @@ <h2>Database Tables</h2>
<li><a href="#ofExtComponentConf">ofExtComponentConf</a></li>
<li><a href="#ofRemoteServerConf">ofRemoteServerConf</a></li>
<li><a href="#ofSecurityAuditLog">ofSecurityAuditLog</a></li>

<li><a href="#ofMucService">ofMucService</a></li>
<li><a href="#ofMucServiceProp">ofMucServiceProp</a></li>
<li><a href="#ofMucRoom">ofMucRoom</a></li>

<li><a href="#ofMucRoomProp">ofMucRoomProp</a></li>
<li><a href="#ofMucRoomRetiree">ofMucRoomRetiree</a></li>
<li><a href="#ofMucAffiliation">ofMucAffiliation</a></li>
Expand All @@ -101,9 +101,10 @@ <h2>Database Tables</h2>
<li><a href="#ofPubsubNodeGroups">ofPubsubNodeGroups</a></li>
<li><a href="#ofPubsubAffiliation">ofPubsubAffiliation</a></li>
<li><a href="#ofPubsubItem">ofPubsubItem</a></li>

<li><a href="#ofPubsubSubscription">ofPubsubSubscription</a></li>
<li><a href="#ofPubsubDefaultConf">ofPubsubDefaultConf</a></li>

<li><a href="#ofSpamReport">ofSpamReport</a></li>
</ul>
</nav>

Expand Down Expand Up @@ -1802,6 +1803,53 @@ <h2>Database Tables</h2>

&nbsp;<a href="#top" class="top">top of page</a>

<table id="ofSpamReport" class="dbtable">
<tbody>
<tr>
<th colspan="4">ofSpamReport (spam rapportage)</th>
</tr>
<tr>
<th>Column Name</th>
<th>Type</th>
<th>Length</th>
<th>Description</th>
</tr>
<tr>
<td>reporter</td>
<td>VARCHAR</td>
<td>1024</td>
<td>JID of entity that is reporting the spam</td>
</tr>
<tr>
<td>reported</td>
<td>VARCHAR</td>
<td>1024</td>
<td>JID of entity that is being reported (the 'offender')</td>
</tr>
<tr>
<td>reason</td>
<td>VARCHAR</td>
<td>255</td>
<td>Identifier (typically a namespace) for the type of spam report</td>
</tr>
<tr>
<td>created</td>
<td>NUMBER</td>
<td></td>
<td>Epoch millis of the creation time of the report</td>
</tr>
<tr>
<td>raw</td>
<td>CLOB</td>
<td></td>
<td>XML representation of the report</td>
</tr>
</tbody>

&nbsp;<a href="#top" class="top">top of page</a>

</table>

</section>

<footer>
Expand Down
7 changes: 7 additions & 0 deletions documentation/openfire.doap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -458,5 +458,12 @@
<xmpp:note xml:lang='en'>Implements most features, except for much of the Destination Address Selection as defined in RFC 6724, Section 6.</xmpp:note>
</xmpp:SupportedXep>
</implements>
<implements>
<xmpp:SupportedXep>
<xmpp:xep rdf:resource="https://xmpp.org/extensions/xep-0377.html"/>
<xmpp:status>complete</xmpp:status>
<xmpp:version>0.3.1</xmpp:version>
</xmpp:SupportedXep>
</implements>
</Project>
</rdf:RDF>
2 changes: 2 additions & 0 deletions documentation/protocol-support.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,8 @@ <h2>List of other XEPs Supported</h2>
<td><a href="https://www.xmpp.org/extensions/xep-0289.html">XEP-0289</a>: Federated MUC for Constrained Environments</td>
</tr><tr>
<td><a href="https://www.xmpp.org/extensions/xep-0321.html">XEP-0321</a>: Remote Roster Management [<a href="#fn15">15</a>]</td>
</tr><tr>
<td><a href="https://www.xmpp.org/extensions/xep-0337.html">XEP-0337</a>: Spam Reporting</td>
</tr><tr>
<td><a href="https://www.xmpp.org/extensions/xep-0359.html">XEP-0359</a>: Unique and Stable Stanza IDs</td>
</tr><tr>
Expand Down
6 changes: 6 additions & 0 deletions i18n/src/main/resources/openfire_i18n.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1347,6 +1347,7 @@ system_property.provider.lockout.className=The class to use to lock-out Openfire
system_property.provider.pubsub-persistence.className=The class used to persist pubsub data.
system_property.provider.pubsub-persistence.caching.delegate-className=The class used by the cache delegate to persist pubsub data.
system_property.provider.securityAudit.className=The class to use to audit actions performed by administrators
system_property.provider.spamReport.className=The class to use for storage of spam reports.
system_property.provider.user.className=The class to use to provide the Openfire users
system_property.provider.vcard.className=The class to use to provide vCard handling
system_property.usermanager.remote-disco-info-timeout-seconds=The maximum time the UserManager should wait, in seconds, for the a remote server to respond to a disco#info request to confirm the presence of a user
Expand Down Expand Up @@ -1479,6 +1480,8 @@ system_property.xmpp.server.resolution-delay=The time to wait for a response for
system_property.xmpp.server.resolution-timeout=The maximum amount of time to wait for successful resolution of a host of a target domain.
system_property.xmpp.server.connection-max-workers=The maximum amount of worker threads attempting to set up a socket connection to a target remote XMPP domain. A value of '1' will effectively make 'Happy Eyeballs' impossible (as that requires concurrent connection attempts).

system_property.spamreport.notify-admins.enabled=Defines if notifications (via XMPP) are to be sent to administrators of the server when a spam report is received.

# Server properties Page

server.properties.title=System Properties
Expand Down Expand Up @@ -4028,3 +4031,6 @@ commands.event.vcardmodified.note.vcard-does-not-exist=VCard not found.
commands.generic.ping.label=Request pong from server
commands.generic.ping.form.title=Server ping result (pong!)
commands.generic.ping.form.field.timestamp.label=Server Time

# Spam Reporting
spamreport.notify-admins.notification-message=User {0} reports receiving spam from {1}. Please review the Openfire admin console (Server > Spam Reports) for more details.
Loading
Loading