DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-2, and 4-21 have been examined in this Non-Final Rejection. Claim 3 has been canceled. Claim 21 is New. Claims 1-2, and 4-21 are currently pending.
Request for Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/10/2026 has been entered.
Priority
Application 18/349,913 was filed on 07/10/2023.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim(s) 1-2, and 4-21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Applicant’s amendments to Claim(s) 1, 11, and 20 added the limitations of "receiving, by a safety gateway, incoming data from the entity that is performing an action at the server, wherein the safety gateway is configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity", which the Examiner finds to be new matter as it does not appear to be supported in the originally filed application. More specifically, ¶0058 in the as filed disclosure describes the safety gateway as an entry point or interface for incoming user data and also responsible for forwarding such data to the detection system for further processing, however, support cannot be found for asynchronously queuing the data.
Claim 21 added the limitations of “wherein the detection service is a scalable microservice configured to operate asynchronously and in parallel such that policy violations are processed with low latency, which the Examiner finds to be new matter as it does not appear to be supported in the originally filed application. More specifically, ¶0079 discloses the operations depicted may be performed in parallel, however support cannot be found for the detection service being a scalable microservice to operate asynchronously such that policy violations are processed with low latency.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, and 4-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Claims 1-2, and 4-21 are directed to a system, method, or product which are/is one of the statutory categories of invention. (Step 1: YES).
Claims 1, 11, and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites a method, and system for monitoring communities, and determining appropriate interventions when violations are determined. For Claims 1, 11 and 20 the limitations of (Claim 1 being representative):
reading a state of […] a history of past policy violations for a plurality of entities […] that supports near real-time communications, wherein the state includes previous classifications of policy violations for an entity of the plurality of entities;
receiving, […], incoming data from the entity that is performing an action […], […] configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity;
determining, there is a new policy violation by the entity […] that supports near real-time communications, wherein the new policy violation is associated with a subsequent classification that indicates a particular type of policy violation, and determined based on the previous classifications of policy violations of the particular type associated with the entity […];
selecting a particular intervention ladder of a plurality of intervention ladders, wherein the particular intervention ladder is specific to the particular type of policy violation
determining a first intervention of a plurality of interventions in the particular intervention ladder for the new policy violation based on the subsequent classification and the previous classifications of the particular type of the policy violations associated with the entity […];
applying the first intervention by an intervention service, wherein the intervention service uses a state of the entity's intervention in applying the first intervention and
storing the new policy violation, the subsequent classification associated with the new policy violation, and the first intervention […],
The above limitations are reciting a policy violation detection process that is considered to be a certain method of organizing human activities. Determining policy violations is a process which one can mitigate risk. The Specification describes determining harmful speech or actions, see paragraph 0017, and 0019. Mitigating risk by limiting harassment, hate speech, bullying, or piracy towards another user falls into the category of being a certain method of organizing human activities. Accordingly, Claims 1, 11 and 20 recite an abstract idea. (Step 2A- Prong 1: YES. The claims are abstract).
This judicial exception is not integrated into a practical application. Claims 1, 11, and 20 recites the additional elements of an immutable database (Claims 1, 11, and 20), a server (Claims 1, 11, and 20), a safety gateway (Claims 1, 11, and 20), a non-transitory computer-readable storage medium (Claims 11), a computer (Claim 11), a processor (Claim 20), and a memory (Claim 20), that implements the identified abstract idea. These additional elements are not described by the applicant and are recited at a high-level of generality (i.e., one or more generic computers performing a generic computer functions) such that it amounts no more than mere instructions to apply the exception using a generic computer components. Accordingly, even in combination these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Claims 1, 11, and 20 are directed to an abstract idea. (Step 2A-Prong 2: NO: the additional claimed elements are not integrated into a practical application).
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of an immutable database (Claims 1, 11, and 20), a server (Claims 1, 11, and 20), a safety gateway (Claims 1, 11, and 20), A non-transitory computer-readable storage medium (Claims 11), a computer (Claim 11), a processor (Claim 20), and a memory (Claim 20), to perform the noted steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept (“significantly more”). Accordingly, even in combination, these additional elements do not provide significantly more. As such claims 1, 11, and 20 are not patent eligible. (Step 2B: NO. The claims do not provide significantly more).
Dependent Claims 2, 4-10, 12-19, and 21 are similarly rejected because they either further define/narrow the abstract idea of independent claims 1, 11 and 20 as discussed above. Claim(s) 13 merely describe(s) the entity being a community of users. Claim(s) 4 & 14 merely describe(s) generating a set of signals associated with actions performed by the entity that supports real-time communications, wherein the new policy violation determination is based on the generated set of signals. Claim(s) 5 & 15 merely describe(s) receiving an appeal indicating that the subsequent classification is incorrect, determining a new classification for the new policy violation or that there was no policy violation, and storing the appeal and the new classification or a correction that there was no policy violation. Claim(s) 6 & 16 merely describe(s) requesting the first intervention to be applied, and storing a log for requesting the first intervention to be applied separately from a log for applying the first intervention. Claim(s) 7 & 17 merely describe(s) indexing classifications by entities. Claim(s) 8 & 18 merely describe(s) selecting the first intervention ladder based on the subsequent classification, determining a current point value associated with the entity and an associated current level on the first intervention ladder based on past policy violation data associated with the previous policy violations, and updating the current point value by adding one or more points based on the new policy violation and updating a next level of the intervention ladder, wherein each increasing level is associated with a harsher intervention action, wherein the first intervention is based on an updated point value. Claim(s) 9 & 19 merely describe(s) storing the updated point value and the next level of the first intervention ladder in association with the entity. Claim(s) 10 merely describe(s) setting an expiration reset time period for the intervention ladder; and after the expiration reset time period has passed without any intervening policy violations associated with the intervention ladder, resetting the updated point value for the entity. Claim(s) 21 merely describes the detection service being a scalable microservice configured to operate asynchronously and in parallel such that policy violations are processed with low latency.
Dependent Claim(s) 2, and 12 recite limitations that further define the abstract idea noted in independent claims 1, 11, and 20. In addition, it recites the additional elements of a machine-learning or rule-based model. The machine-learning or rule-based model, are recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computing component. Even in combination, these additional elements do not integrate the abstract idea into a practical application and do not amount to significantly more than the abstract idea itself. Claims 4-7, 9, 14-17, and 19 include the additional elements of an immutable database, and server. The immutable database, and server are analyzed in the same manner as the immutable database, and server in the independent claim and do not provide a practical application or significantly more for the same reasons above. Therefore claims 2, 4-10, 12-19, and 21 are considered patent ineligible for the reasons given above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-7, 11, 13-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Leliwa (US 20200267165 A1), in view of Chen (US20220198468), and in further view of Comartin (Asynchronous request-response pattern for non-blocking workflows ).
Regarding Claim 1,
Leliwa discloses, (Currently Amended) A computer-implemented method comprising: reading a state of an immutable database that stores a history of past policy violations for a plurality of entities at a server that supports near real-time communications, “The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned” (Leliwa Par. 0131). “The development of the Internet—among many undeniable benefits—is contributing to the proliferation of new threats for its users, especially kids and online communities. Such communities can express themselves, hang out and have conversations using online services such as messengers, chatrooms, forums, discussion websites, photo and video sharing services, social networking services, and so on. The threats come from other Internet users who act against healthy conversations for a variety of reasons. Online violence (or cyberviolence) is one of the most common undesirable behaviors within online communities, whereas the most common method for combating it is content moderation. Furthermore, online violence is one of the primary reasons why users leave online communities and it contributes (especially cyberbullying) to much more dangerous effects, including suicides among children and youth” (Leliwa Par. 0002). “FIG. 4 is a diagram illustrating the process of performing a single intervention according to an embodiment. The diagram shows an exemplary exchange of messages between three users that can be identified with their IDs: USER#2425, USER#3732, USER#1163. This could be a regular conversation using either a chat or a forum. The messages appear chronologically from the top to the bottom. The first message written by USER#2425 is sent to Intervention System 110 and then to Online Violence Detection System 130, where it is classified as not containing online violence. There is no system reaction at this point. The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
wherein the state includes previous classifications of policy violations for an entity of the plurality of entities; “The configuration described in FIG. 7 allows two types of penalties: temporary ban and permanent ban. The script verifies if the user was banned before and—if the test is positive—it adds 2 to the number of interventions obtained by the user within the predefined time period. Then, based on that number, the configuration sends a request to the last module of Intervention System 110” (Leliwa Par. 0137). "The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). “FIG. 4 is a diagram illustrating the process of performing a single intervention according to an embodiment. The diagram shows an exemplary exchange of messages between three users that can be identified with their IDs: USER#2425, USER#3732, USER#1163. This could be a regular conversation using either a chat or a forum. The messages appear chronologically from the top to the bottom. The first message written by USER#2425 is sent to Intervention System 110 and then to Online Violence Detection System 130, where it is classified as not containing online violence. There is no system reaction at this point. The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
receiving, by a safety gateway, incoming data from the entity that is performing an action at the server, "Message Analyzer 114A is a module responsible for sending requests for and receiving messages and conversations from Online Community 140. The most recommended and convenient method of communication with Online Community 140 is to use its API 140B that allows developers to interact with Service 140A, e.g. reading and sending messages, creating and authorizing accounts, performing and automating moderators' actions. Most of the biggest online communities use APIs that their partners can be provided with. Many online communities offer access to their public APIs. In an embodiment, Message Analyzer 114A communicates with Online Community 140 using its API 140B. In other embodiment, Intervention System 110 is installed on the client's servers and integrated on-premise directly with client's Service 140A" (Leliwa Par. 0098).
there is a new policy violation by the entity at the server that supports near real-time communications, “1. Chat offers a real-time transmission of text messages from sender to receiver (or receivers in one-to-many group chats). This type is typical for a range of chat services, including messaging apps and platforms as well as dedicated chats on websites and services, including chats on streaming and content sharing platforms and various customer support/help desk services. For this type of communication, the primary form of message is a text message sent within the same chatroom (or other organizational unit) where the violent user message was sent. The secondary form is a private or direct message sent directly to the violent user (not visible by other users). 2. Forum offers a conversation in the form of posted messages. The main difference between forum and chat is that forum messages at least temporarily archived and available for the users. Also, forum messages are often longer than chat messages. Forums can be organized in more complex hierarchical manner, e.g. posts (original and following), comments-to-posts and comments-to-comments. For content sharing platforms, a video or an image with description can be treated as a post. This type is represented by online forums, message boards, image boards, discussion websites, social networking services and content sharing apps and platforms. For this type of communication, the primary form of message is a post or comment sent as a reply to the post or comment sent by the violent user. The secondary form is similar to the chat form—a private or direct message sent to the violent user and not visible by other users” (Leliwa Par. 0031-0032). “The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
wherein the new policy violation is associated with a subsequent classification that indicates a particular type of policy violation, and determined based on the previous classifications of policy violations of the particular type associated with the entity in the database; "System Databases 112 and System Processors 114 can also be distributed geographically in the known manner. Intervention System 110 uses Online Violence Detection System 130 in order to verify whether or not input text contains online violence and to determine online violence categories" (Leliwa Par. 0023). "The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). "Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message" (Leliwa Par. 0121-0127).
selecting a particular intervention ladder of a plurality of intervention ladders, wherein the particular intervention ladder is specific to the particular type of policy violation "It uses responses generated by Intervention System 110 based on type and severity of detected online violence and knowledge about particular violent user and online community" (Leliwa Par. 0043). "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 starts with a violence detection. The script verifies how many interventions the user got within a predefined time period prior to the current intervention. In an embodiment, the time period can be defined for the whole community as well as for its particular communication channels individually. Defining the time period is particularly important for fast-paced conversations in order to not exaggerate punishing for overdue offenses. For example, if the time period is defined as one hour and the user got interventions at 10:05 am, 10:23 am, 10:48 am and the current intervention was sent at 11:14 am, the first intervention at 10:05 am is overdue and therefore the user got only two interventions prior to the current intervention within the time period. The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service" (Leliwa Par. 0133-0136).
determining a first intervention of a plurality of interventions in the particular intervention ladder for the new policy violation based on the subsequent classification and the previous classifications of the particular type of the policy violations "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 allows two types of penalties: temporary ban and permanent ban. The script verifies if the user was banned before and—if the test is positive—it adds 2 to the number of interventions obtained by the user within the predefined time period. Then, based on that number, the configuration sends a request to the last module of Intervention System 110. If the final number of interventions is: 0, a request for empathetic intervention is sent; 1, a request for normative soft intervention is sent; 2, a request for normative hard intervention is sent; more than 2 and the user was not previously banned, a request for temporary ban is sent; more than 2 and the user was previously banned, a request for permanent ban is sent" (Leliwa Par. 0137-0142). The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131).
associated with the entity in the database; "The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). "Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message" (Leliwa Par. 0121-0127).
applying the first intervention by an intervention service, wherein the intervention service uses a state of the entity's intervention in applying the first intervention "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 starts with a violence detection. The script verifies how many interventions the user got within a predefined time period prior to the current intervention. In an embodiment, the time period can be defined for the whole community as well as for its particular communication channels individually. Defining the time period is particularly important for fast-paced conversations in order to not exaggerate punishing for overdue offenses. For example, if the time period is defined as one hour and the user got interventions at 10:05 am, 10:23 am, 10:48 am and the current intervention was sent at 11:14 am, the first intervention at 10:05 am is overdue and therefore the user got only two interventions prior to the current intervention within the time period. The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service" (Leliwa Par. 0133-0136). "A life cycle of using Intervention System 110 within Online Community 140 largely depends on the amount of collected data. Therefore, it is usually the most effective to start off with rule-based and algorithmic approaches. Then, as the amount of collected data grows, it is reasonable to follow up with a hybrid approach introducing more and more statistical approaches. A mature integration should utilize a hybrid approach reinforced with very advanced statistical approaches that can truly benefit from large datasets. An example of introducing a hybrid approach to the diagram described in FIG. 7 is to keep the symbolic methods for determining when to send the interventions and to apply statistical classifiers for choosing what intervention should be sent based on all user-related data available in Community Database 112A" (Leliwa Par. 0146).
storing the new policy violation, the subsequent classification associated with the new policy violation, and the first intervention in the database “Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message” (Leliwa Par. 0121-0127).
Leliwa discloses reading a state of violations of a database that stores a history of past policy violations for a plurality of entities at a server that supports real-time communication, determining a new policy violation by the entity at the server, where the new policy violation is associated with a classification that indicates a type of policy violation and is determined on the previous policy violations, selecting a first intervention ladder of a plurality of intervention ladders specific to the first type of policy violation, determining an intervention for the policy violation based on the classification and previous policy violations, applying the intervention by an intervention services, and storing the new policy violation, classification, and intervention in a database. Leliwa, fails to disclose an immutable database. Chen, however, does disclose,
the immutable database. “For ease of management of the blockchain, the violation credit record data of the user may be stored into a corresponding blockchain according to the user identifier information of the user. The violation credit record data in the blockchain is stored in the form of a key-value pair, hash operation may be performed according to the user identifier information to obtain a hash value, and hash operation may be performed on a key element in block data to obtain a hash value. If the hash values obtained through two operation processes are consistent, the blockchain is the blockchain corresponding to the user” (Chen Par. 0121-0122). “Blockchain: the blockchain technology is derived from the Bitcoin technology, is an underlying technology of Bitcoin, and is a decentralized distributed ledger database. The blockchain is a string of data blocks (that is, blocks) generated through association by using a cryptographic algorithm, and each data block includes information that is effectively confirmed by a plurality of blockchain network transactions. Based on this, cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured. To clearly describe a blockchain system and a chain formed by blocks in the following description, a blockchain system including a plurality of nodes may be referred to as a blockchain network, and a chain formed by data stored in the blockchain is referred to as a blockchain” (Chen Par. 0033).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of classifying and intervening in online violations of user activity of Leliwa with an immutable database of Chen because cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured (Chen Par. 0033).
The combination of Leliwa and Chen disclose determining policy violations, and applying actions using an immutable database, but fail to disclose wherein the safety gateway is configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity. Comartin discloses an asynchronous request-response pattern for messages. Comartin further discloses, wherein the safety gateway is configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity; "The asynchronous request-response pattern allows you to tell a sender that the message has been processed and what the outcome or result was. You can leverage this to then build workflows to involve many different services all in a non-blocking way" (Comartin Page 3).
PNG
media_image1.png
916
788
media_image1.png
Greyscale
PNG
media_image2.png
1008
794
media_image2.png
Greyscale
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of classifying and intervening in online violations of user activity of Leliwa and Chen with asynchronously queuing and forwarding the incoming data to a detection service without blocking or serializing the action performed by the entity of Comartin to involve many different services all in a non-blocking way (Comartin Page 3).
Regarding Claim 11,
Leliwa discloses, read a state of an immutable database that stores a history of past policy violations for a plurality of entities at a server that supports near real-time communications, “The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned” (Leliwa Par. 0131). “The development of the Internet—among many undeniable benefits—is contributing to the proliferation of new threats for its users, especially kids and online communities. Such communities can express themselves, hang out and have conversations using online services such as messengers, chatrooms, forums, discussion websites, photo and video sharing services, social networking services, and so on. The threats come from other Internet users who act against healthy conversations for a variety of reasons. Online violence (or cyberviolence) is one of the most common undesirable behaviors within online communities, whereas the most common method for combating it is content moderation. Furthermore, online violence is one of the primary reasons why users leave online communities and it contributes (especially cyberbullying) to much more dangerous effects, including suicides among children and youth” (Leliwa Par. 0002). “FIG. 4 is a diagram illustrating the process of performing a single intervention according to an embodiment. The diagram shows an exemplary exchange of messages between three users that can be identified with their IDs: USER#2425, USER#3732, USER#1163. This could be a regular conversation using either a chat or a forum. The messages appear chronologically from the top to the bottom. The first message written by USER#2425 is sent to Intervention System 110 and then to Online Violence Detection System 130, where it is classified as not containing online violence. There is no system reaction at this point. The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
wherein the state includes previous classifications of policy violations for an entity of the plurality of entities; “The configuration described in FIG. 7 allows two types of penalties: temporary ban and permanent ban. The script verifies if the user was banned before and—if the test is positive—it adds 2 to the number of interventions obtained by the user within the predefined time period. Then, based on that number, the configuration sends a request to the last module of Intervention System 110” (Leliwa Par. 0137). "The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). “FIG. 4 is a diagram illustrating the process of performing a single intervention according to an embodiment. The diagram shows an exemplary exchange of messages between three users that can be identified with their IDs: USER#2425, USER#3732, USER#1163. This could be a regular conversation using either a chat or a forum. The messages appear chronologically from the top to the bottom. The first message written by USER#2425 is sent to Intervention System 110 and then to Online Violence Detection System 130, where it is classified as not containing online violence. There is no system reaction at this point. The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
receiving, by a safety gateway, incoming data from the entity that is performing an action at the server, "Message Analyzer 114A is a module responsible for sending requests for and receiving messages and conversations from Online Community 140. The most recommended and convenient method of communication with Online Community 140 is to use its API 140B that allows developers to interact with Service 140A, e.g. reading and sending messages, creating and authorizing accounts, performing and automating moderators' actions. Most of the biggest online communities use APIs that their partners can be provided with. Many online communities offer access to their public APIs. In an embodiment, Message Analyzer 114A communicates with Online Community 140 using its API 140B. In other embodiment, Intervention System 110 is installed on the client's servers and integrated on-premise directly with client's Service 140A" (Leliwa Par. 0098).
there is a new policy violation by the entity at the server that supports near real-time communications, “1. Chat offers a real-time transmission of text messages from sender to receiver (or receivers in one-to-many group chats). This type is typical for a range of chat services, including messaging apps and platforms as well as dedicated chats on websites and services, including chats on streaming and content sharing platforms and various customer support/help desk services. For this type of communication, the primary form of message is a text message sent within the same chatroom (or other organizational unit) where the violent user message was sent. The secondary form is a private or direct message sent directly to the violent user (not visible by other users). 2. Forum offers a conversation in the form of posted messages. The main difference between forum and chat is that forum messages at least temporarily archived and available for the users. Also, forum messages are often longer than chat messages. Forums can be organized in more complex hierarchical manner, e.g. posts (original and following), comments-to-posts and comments-to-comments. For content sharing platforms, a video or an image with description can be treated as a post. This type is represented by online forums, message boards, image boards, discussion websites, social networking services and content sharing apps and platforms. For this type of communication, the primary form of message is a post or comment sent as a reply to the post or comment sent by the violent user. The secondary form is similar to the chat form—a private or direct message sent to the violent user and not visible by other users” (Leliwa Par. 0031-0032). “The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
wherein the new policy violation is associated with a subsequent classification that indicates a particular type of policy violation, and determined based on the previous classifications of policy violations of the particular type associated with the entity in the database; "System Databases 112 and System Processors 114 can also be distributed geographically in the known manner. Intervention System 110 uses Online Violence Detection System 130 in order to verify whether or not input text contains online violence and to determine online violence categories" (Leliwa Par. 0023). "The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). "Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message" (Leliwa Par. 0121-0127).
select a particular intervention ladder of the plurality of intervention ladders, wherein the particular intervention ladder is specific to the first type of policy violation; "It uses responses generated by Intervention System 110 based on type and severity of detected online violence and knowledge about particular violent user and online community" (Leliwa Par. 0043). "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 starts with a violence detection. The script verifies how many interventions the user got within a predefined time period prior to the current intervention. In an embodiment, the time period can be defined for the whole community as well as for its particular communication channels individually. Defining the time period is particularly important for fast-paced conversations in order to not exaggerate punishing for overdue offenses. For example, if the time period is defined as one hour and the user got interventions at 10:05 am, 10:23 am, 10:48 am and the current intervention was sent at 11:14 am, the first intervention at 10:05 am is overdue and therefore the user got only two interventions prior to the current intervention within the time period. The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service" (Leliwa Par. 0133-0136).
determine a first intervention of a plurality of interventions in the first intervention ladder for the new policy violation based on the subsequent classification and the previous classifications of the particular type of the policy violations "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 allows two types of penalties: temporary ban and permanent ban. The script verifies if the user was banned before and—if the test is positive—it adds 2 to the number of interventions obtained by the user within the predefined time period. Then, based on that number, the configuration sends a request to the last module of Intervention System 110. If the final number of interventions is: 0, a request for empathetic intervention is sent; 1, a request for normative soft intervention is sent; 2, a request for normative hard intervention is sent; more than 2 and the user was not previously banned, a request for temporary ban is sent; more than 2 and the user was previously banned, a request for permanent ban is sent" (Leliwa Par. 0137-0142). The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131).
associated with the entity in the database; "The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). "Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message" (Leliwa Par. 0121-0127).
apply the first intervention by an intervention service, wherein the intervention service uses a state of the entity's intervention in applying the first intervention; and "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 starts with a violence detection. The script verifies how many interventions the user got within a predefined time period prior to the current intervention. In an embodiment, the time period can be defined for the whole community as well as for its particular communication channels individually. Defining the time period is particularly important for fast-paced conversations in order to not exaggerate punishing for overdue offenses. For example, if the time period is defined as one hour and the user got interventions at 10:05 am, 10:23 am, 10:48 am and the current intervention was sent at 11:14 am, the first intervention at 10:05 am is overdue and therefore the user got only two interventions prior to the current intervention within the time period. The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service" (Leliwa Par. 0133-0136). "A life cycle of using Intervention System 110 within Online Community 140 largely depends on the amount of collected data. Therefore, it is usually the most effective to start off with rule-based and algorithmic approaches. Then, as the amount of collected data grows, it is reasonable to follow up with a hybrid approach introducing more and more statistical approaches. A mature integration should utilize a hybrid approach reinforced with very advanced statistical approaches that can truly benefit from large datasets. An example of introducing a hybrid approach to the diagram described in FIG. 7 is to keep the symbolic methods for determining when to send the interventions and to apply statistical classifiers for choosing what intervention should be sent based on all user-related data available in Community Database 112A" (Leliwa Par. 0146).
store the new policy violation, the subsequent classification associated with the new policy violation, and the first intervention in the database. “Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message” (Leliwa Par. 0121-0127).
Leliwa discloses reading a state of violations of a database that stores a history of past policy violations for a plurality of entities at a server that supports real-time communication, determining a new policy violation by the entity at the server, where the new policy violation is associated with a classification that indicates a type of policy violation and is determined on the previous policy violations, selecting a first intervention ladder of a plurality of intervention ladders specific to the first type of policy violation, determining an intervention for the policy violation based on the classification and previous policy violations, applying the intervention by an intervention services, and storing the new policy violation, classification, and intervention in a database. Leliwa, fails to disclose a non-transitory computer-readable storage medium and an immutable database. Chen, however, does disclose,
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to: (Chen Par. 0201-0202)
the immutable database. “For ease of management of the blockchain, the violation credit record data of the user may be stored into a corresponding blockchain according to the user identifier information of the user. The violation credit record data in the blockchain is stored in the form of a key-value pair, hash operation may be performed according to the user identifier information to obtain a hash value, and hash operation may be performed on a key element in block data to obtain a hash value. If the hash values obtained through two operation processes are consistent, the blockchain is the blockchain corresponding to the user” (Chen Par. 0121-0122). “Blockchain: the blockchain technology is derived from the Bitcoin technology, is an underlying technology of Bitcoin, and is a decentralized distributed ledger database. The blockchain is a string of data blocks (that is, blocks) generated through association by using a cryptographic algorithm, and each data block includes information that is effectively confirmed by a plurality of blockchain network transactions. Based on this, cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured. To clearly describe a blockchain system and a chain formed by blocks in the following description, a blockchain system including a plurality of nodes may be referred to as a blockchain network, and a chain formed by data stored in the blockchain is referred to as a blockchain” (Chen Par. 0033).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of classifying and intervening in online violations of user activity of Leliwa with a memory, processor, and immutable database of Chen because cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured (Chen Par. 0033).
The combination of Leliwa and Chen disclose determining policy violations, and applying actions using an immutable database, but fail to disclose wherein the safety gateway is configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity. Comartin discloses an asynchronous request-response pattern for messages. Comartin further discloses, wherein the safety gateway is configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity; "The asynchronous request-response pattern allows you to tell a sender that the message has been processed and what the outcome or result was. You can leverage this to then build workflows to involve many different services all in a non-blocking way" (Comartin Page 3).
PNG
media_image1.png
916
788
media_image1.png
Greyscale
PNG
media_image2.png
1008
794
media_image2.png
Greyscale
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of classifying and intervening in online violations of user activity of Leliwa and Chen with asynchronously queuing and forwarding the incoming data to a detection service without blocking or serializing the action performed by the entity of Comartin to involve many different services all in a non-blocking way (Comartin Page 3).
Regarding Claim 20,
Leliwa discloses, A system comprising: (Leliwa Abstract)
read a state of an immutable database that stores a history of past policy violations for a plurality of entities at a server that supports near real-time communications, “The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned” (Leliwa Par. 0131). “The development of the Internet—among many undeniable benefits—is contributing to the proliferation of new threats for its users, especially kids and online communities. Such communities can express themselves, hang out and have conversations using online services such as messengers, chatrooms, forums, discussion websites, photo and video sharing services, social networking services, and so on. The threats come from other Internet users who act against healthy conversations for a variety of reasons. Online violence (or cyberviolence) is one of the most common undesirable behaviors within online communities, whereas the most common method for combating it is content moderation. Furthermore, online violence is one of the primary reasons why users leave online communities and it contributes (especially cyberbullying) to much more dangerous effects, including suicides among children and youth” (Leliwa Par. 0002). “FIG. 4 is a diagram illustrating the process of performing a single intervention according to an embodiment. The diagram shows an exemplary exchange of messages between three users that can be identified with their IDs: USER#2425, USER#3732, USER#1163. This could be a regular conversation using either a chat or a forum. The messages appear chronologically from the top to the bottom. The first message written by USER#2425 is sent to Intervention System 110 and then to Online Violence Detection System 130, where it is classified as not containing online violence. There is no system reaction at this point. The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
wherein the state includes previous classifications of policy violations for an entity of the plurality of entities at the server that supports near real-time communications; “The configuration described in FIG. 7 allows two types of penalties: temporary ban and permanent ban. The script verifies if the user was banned before and—if the test is positive—it adds 2 to the number of interventions obtained by the user within the predefined time period. Then, based on that number, the configuration sends a request to the last module of Intervention System 110” (Leliwa Par. 0137). "The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). “FIG. 4 is a diagram illustrating the process of performing a single intervention according to an embodiment. The diagram shows an exemplary exchange of messages between three users that can be identified with their IDs: USER#2425, USER#3732, USER#1163. This could be a regular conversation using either a chat or a forum. The messages appear chronologically from the top to the bottom. The first message written by USER#2425 is sent to Intervention System 110 and then to Online Violence Detection System 130, where it is classified as not containing online violence. There is no system reaction at this point. The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
receiving, by a safety gateway, incoming data from the entity that is performing an action at the server, "Message Analyzer 114A is a module responsible for sending requests for and receiving messages and conversations from Online Community 140. The most recommended and convenient method of communication with Online Community 140 is to use its API 140B that allows developers to interact with Service 140A, e.g. reading and sending messages, creating and authorizing accounts, performing and automating moderators' actions. Most of the biggest online communities use APIs that their partners can be provided with. Many online communities offer access to their public APIs. In an embodiment, Message Analyzer 114A communicates with Online Community 140 using its API 140B. In other embodiment, Intervention System 110 is installed on the client's servers and integrated on-premise directly with client's Service 140A" (Leliwa Par. 0098).
there is a new policy violation by the entity at the server that supports near real-time communications, “1. Chat offers a real-time transmission of text messages from sender to receiver (or receivers in one-to-many group chats). This type is typical for a range of chat services, including messaging apps and platforms as well as dedicated chats on websites and services, including chats on streaming and content sharing platforms and various customer support/help desk services. For this type of communication, the primary form of message is a text message sent within the same chatroom (or other organizational unit) where the violent user message was sent. The secondary form is a private or direct message sent directly to the violent user (not visible by other users). 2. Forum offers a conversation in the form of posted messages. The main difference between forum and chat is that forum messages at least temporarily archived and available for the users. Also, forum messages are often longer than chat messages. Forums can be organized in more complex hierarchical manner, e.g. posts (original and following), comments-to-posts and comments-to-comments. For content sharing platforms, a video or an image with description can be treated as a post. This type is represented by online forums, message boards, image boards, discussion websites, social networking services and content sharing apps and platforms. For this type of communication, the primary form of message is a post or comment sent as a reply to the post or comment sent by the violent user. The secondary form is similar to the chat form—a private or direct message sent to the violent user and not visible by other users” (Leliwa Par. 0031-0032). “The second message from USER#3732 is also sent to Intervention System 110 and Online Violence Detection System 130, where it is classified as online violence. USER#1163 is a concealed chatter bot controlled by Intervention System 110. The violent message detection triggers an autonomous reaction: USER#1163 replies to the violent comment with an intervention from one of predefined intervention groups. In this case, the system sends a utilitarian message that refers to a utilitarian perspective showing how the discussion could be more fruitful and pleasurable for all under specific conditions” (Leliwa Par. 0077).
wherein the new policy violation is associated with a subsequent classification that indicates a particular type of policy violation, and determined based on the previous classifications of policy violations of the particular type associated with the entity in the database; "System Databases 112 and System Processors 114 can also be distributed geographically in the known manner. Intervention System 110 uses Online Violence Detection System 130 in order to verify whether or not input text contains online violence and to determine online violence categories" (Leliwa Par. 0023). "The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). "Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message" (Leliwa Par. 0121-0127).
select a particular intervention ladder of the plurality of intervention ladders, wherein the particular intervention ladder is specific to the particular type of policy violation; "It uses responses generated by Intervention System 110 based on type and severity of detected online violence and knowledge about particular violent user and online community" (Leliwa Par. 0043). "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 starts with a violence detection. The script verifies how many interventions the user got within a predefined time period prior to the current intervention. In an embodiment, the time period can be defined for the whole community as well as for its particular communication channels individually. Defining the time period is particularly important for fast-paced conversations in order to not exaggerate punishing for overdue offenses. For example, if the time period is defined as one hour and the user got interventions at 10:05 am, 10:23 am, 10:48 am and the current intervention was sent at 11:14 am, the first intervention at 10:05 am is overdue and therefore the user got only two interventions prior to the current intervention within the time period. The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service" (Leliwa Par. 0133-0136).
determine a first intervention of a plurality of interventions in the particular intervention ladder for the new policy violation based on the subsequent classification and the previous classifications of the particular type of the policy violations "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 allows two types of penalties: temporary ban and permanent ban. The script verifies if the user was banned before and—if the test is positive—it adds 2 to the number of interventions obtained by the user within the predefined time period. Then, based on that number, the configuration sends a request to the last module of Intervention System 110. If the final number of interventions is: 0, a request for empathetic intervention is sent; 1, a request for normative soft intervention is sent; 2, a request for normative hard intervention is sent; more than 2 and the user was not previously banned, a request for temporary ban is sent; more than 2 and the user was previously banned, a request for permanent ban is sent" (Leliwa Par. 0137-0142). The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131).
associated with the entity in the database; "The configuration has access to and can utilize any information delivered in Message Analyzer Output 560 and stored in Community Database 112A. The configuration presented in FIG. 7 utilizes only information about previous interventions of the user and whether or not the user was previously banned. The required calculations and operations can be performed using the configuration script. For example, if Community Database 112A contains only entries describing previous interventions, the number of all interventions can be calculated in the script as a number of those entries" (Leliwa Par. 0131). "Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message" (Leliwa Par. 0121-0127).
apply the first intervention by an intervention service, wherein the intervention service uses a state of the entity's intervention in applying the first intervention; and "Another feature of Online Violence Detection System 130 is in-depth categorization of online violence phenomena. Different types of online violence requires different types of reactions. For example, the best reaction to mild personal attack is often an empathetic intervention, whereas sexual harassment usually require a strong disapproval. In general, the more granular categorization, the better possibilities to assign proper reaction to detected messages. Ability to extract certain words and phrases related to online violence is another valuable feature as it can be used to generate a better intervention that precisely points out its rationale. For example, if a personal attack is detected because one user called another user an idiot, the intervention can point out that calling other users idiots is not accepted within this community. Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109). "The configuration described in FIG. 7 starts with a violence detection. The script verifies how many interventions the user got within a predefined time period prior to the current intervention. In an embodiment, the time period can be defined for the whole community as well as for its particular communication channels individually. Defining the time period is particularly important for fast-paced conversations in order to not exaggerate punishing for overdue offenses. For example, if the time period is defined as one hour and the user got interventions at 10:05 am, 10:23 am, 10:48 am and the current intervention was sent at 11:14 am, the first intervention at 10:05 am is overdue and therefore the user got only two interventions prior to the current intervention within the time period. The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service" (Leliwa Par. 0133-0136). "A life cycle of using Intervention System 110 within Online Community 140 largely depends on the amount of collected data. Therefore, it is usually the most effective to start off with rule-based and algorithmic approaches. Then, as the amount of collected data grows, it is reasonable to follow up with a hybrid approach introducing more and more statistical approaches. A mature integration should utilize a hybrid approach reinforced with very advanced statistical approaches that can truly benefit from large datasets. An example of introducing a hybrid approach to the diagram described in FIG. 7 is to keep the symbolic methods for determining when to send the interventions and to apply statistical classifiers for choosing what intervention should be sent based on all user-related data available in Community Database 112A" (Leliwa Par. 0146).
store the new policy violation, the subsequent classification associated with the new policy violation, and the first intervention in the database. “Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message” (Leliwa Par. 0121-0127).
Leliwa discloses reading a state of violations of a database that stores a history of past policy violations for a plurality of entities at a server that supports real-time communication, determining a new policy violation by the entity at the server, where the new policy violation is associated with a classification that indicates a type of policy violation and is determined on the previous policy violations, selecting a first intervention ladder of a plurality of intervention ladders specific to the first type of policy violation, determining an intervention for the policy violation based on the classification and previous policy violations, applying the intervention by an intervention services, and storing the new policy violation, classification, and intervention in a database. Leliwa, fails to explicitly disclose a processor, memory, and immutable database. Chen, however, does disclose,
a processor; and a memory storing instructions that, when executed by the processor, configure the system to: “A speech information communication management apparatus may be provided, including: at least one memory configured to store computer program code; and at least one processor configured to read the computer program code and operated as instructed by the computer program code, the computer program code comprising” (Chen Par. 0009).
an immutable database “For ease of management of the blockchain, the violation credit record data of the user may be stored into a corresponding blockchain according to the user identifier information of the user. The violation credit record data in the blockchain is stored in the form of a key-value pair, hash operation may be performed according to the user identifier information to obtain a hash value, and hash operation may be performed on a key element in block data to obtain a hash value. If the hash values obtained through two operation processes are consistent, the blockchain is the blockchain corresponding to the user” (Chen Par. 0121-0122). “Blockchain: the blockchain technology is derived from the Bitcoin technology, is an underlying technology of Bitcoin, and is a decentralized distributed ledger database. The blockchain is a string of data blocks (that is, blocks) generated through association by using a cryptographic algorithm, and each data block includes information that is effectively confirmed by a plurality of blockchain network transactions. Based on this, cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured. To clearly describe a blockchain system and a chain formed by blocks in the following description, a blockchain system including a plurality of nodes may be referred to as a blockchain network, and a chain formed by data stored in the blockchain is referred to as a blockchain” (Chen Par. 0033).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of classifying and intervening in online violations of user activity of Leliwa with a memory, processor, and immutable database of Chen because cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured (Chen Par. 0033).
The combination of Leliwa and Chen disclose determining policy violations, and applying actions using an immutable database, but fail to disclose wherein the safety gateway is configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity. Comartin discloses an asynchronous request-response pattern for messages. Comartin further discloses, wherein the safety gateway is configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity; "The asynchronous request-response pattern allows you to tell a sender that the message has been processed and what the outcome or result was. You can leverage this to then build workflows to involve many different services all in a non-blocking way" (Comartin Page 3).
PNG
media_image1.png
916
788
media_image1.png
Greyscale
PNG
media_image2.png
1008
794
media_image2.png
Greyscale
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of classifying and intervening in online violations of user activity of Leliwa and Chen with asynchronously queuing and forwarding the incoming data to a detection service without blocking or serializing the action performed by the entity of Comartin to involve many different services all in a non-blocking way (Comartin Page 3).
Regarding Claim 13
The combination of Leliwa, Chen, and Comartin disclose the method of Claim 11, as shown above. Leliwa further discloses, wherein the entity is a community of users. "Other Systems/Applications 160 are systems, including commercial and non-commercial systems, and associated software applications that cannot be perceived as Online Communities 140 but still have the capability to access and use Intervention System 110 through one or more application programming interfaces (APIs) as further described below. For the sake of clarity, online community can be defined as any group of people who discuss anything using the Internet as a medium of communication. Therefore, even people who know each other in real life (e.g. friends from college or co-workers) can be treated as an online community while using any instant messaging platform or service" (Leliwa Par. 0025).
Regarding Claim 4, and Claim 14,
The combination of Leliwa, Chen, and Comartin disclose the method of Claim 1, and computer-readable storage medium of claim 11, as shown above. Leliwa further discloses, generating a set of signals associated with actions performed by the entity at the server that supports near real-time communications, wherein the new policy violation determination is based on the generated set of signals. "The configuration described in FIG. 7 starts with a violence detection. The script verifies how many interventions the user got within a predefined time period prior to the current intervention. In an embodiment, the time period can be defined for the whole community as well as for its particular communication channels individually. Defining the time period is particularly important for fast-paced conversations in order to not exaggerate punishing for overdue offenses. For example, if the time period is defined as one hour and the user got interventions at 10:05 am, 10:23 am, 10:48 am and the current intervention was sent at 11:14 am, the first intervention at 10:05 am is overdue and therefore the user got only two interventions prior to the current intervention within the time period. The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service. The configuration described in FIG. 7 allows two types of penalties: temporary ban and permanent ban. The script verifies if the user was banned before and—if the test is positive—it adds 2 to the number of interventions obtained by the user within the predefined time period. Then, based on that number, the configuration sends a request to the last module of Intervention System 110. If the final number of interventions is: 0, a request for empathetic intervention is sent; 1, a request for normative soft intervention is sent; 2, a request for normative hard intervention is sent; more than 2 and the user was not previously banned, a request for temporary ban is sent; more than 2 and the user was previously banned, a request for permanent ban is sent" (Leliwa Par. 0132-0142).
Regarding Claim 5, and Claim 15,
The combination of Leliwa, Chen, and Comartin disclose the method of Claim 1, as shown above. Leliwa further discloses, The computer-implemented method of claim 1, further comprising: receiving an appeal indicating that the subsequent classification is incorrect; "For example, if Online Community 140 allows its users to rate any message with positive or negative points (upvote and downvote), it can be used to evaluate how an intervention was accepted by other users. Positive points can indicate that the intervention was appropriate, where negative points can signa bad intervention or even a false positive in terms of online violence detection." (Leliwa Par. 0129).
determining a new classification for the new policy violation or that there was no policy violation; and Positive points can indicate that the intervention was appropriate, where negative points can signa bad intervention or even a false positive in terms of online violence detection." (Leliwa Par. 0129).
storing, in the immutable database, the appeal and the new classification or a correction that there was no policy violation. "There is also another important feature that can be used to evaluate performed interventions and in turn to provide better interventions in the future. If Online Community 140 utilizes community points or any other form of awarding good contributions, Message Analyzer 114A can proactively request Online Community 140 for such information regarding the intervention message. It can be performed for a predefined period of time in regular intervals. This information can be passed through the following modules of Intervention System 110 and stored in proper databases in order to increase chances of providing good interventions in the future. For example, if Online Community 140 allows its users to rate any message with positive or negative points (upvote and downvote), it can be used to evaluate how an intervention was accepted by other users. Positive points can indicate that the intervention was appropriate, where negative points can signa bad intervention or even a false positive in terms of online violence detection." (Leliwa Par. 0129).
Leliwa discloses receiving an appeal indicating that the determined classification is incorrect, determining a new classification for the new policy violation or that there was no policy violation; and storing, in the immutable database, the appeal and the new classification or a correction that there was no policy violation. Leliwa fails to disclose the database being immutable. Chen, however, does disclose,
an immutable database “For ease of management of the blockchain, the violation credit record data of the user may be stored into a corresponding blockchain according to the user identifier information of the user. The violation credit record data in the blockchain is stored in the form of a key-value pair, hash operation may be performed according to the user identifier information to obtain a hash value, and hash operation may be performed on a key element in block data to obtain a hash value. If the hash values obtained through two operation processes are consistent, the blockchain is the blockchain corresponding to the user” (Chen Par. 0121-0122). “Blockchain: the blockchain technology is derived from the Bitcoin technology, is an underlying technology of Bitcoin, and is a decentralized distributed ledger database. The blockchain is a string of data blocks (that is, blocks) generated through association by using a cryptographic algorithm, and each data block includes information that is effectively confirmed by a plurality of blockchain network transactions. Based on this, cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured. To clearly describe a blockchain system and a chain formed by blocks in the following description, a blockchain system including a plurality of nodes may be referred to as a blockchain network, and a chain formed by data stored in the blockchain is referred to as a blockchain” (Chen Par. 0033).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of receiving an appeal indicating that the determined classification is incorrect, determining a new classification for the new policy violation or that there was no policy violation; and storing, in the immutable database, the appeal and the new classification or a correction that there was no policy violation of Leliwa and Chen with an immutable database of Chen because cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured (Chen Par. 0033).
Regarding Claim 6, and Claim 16,
The combination of Leliwa, Chen, and Comartin disclose the method of Claim 1, computer-readable storage medium of claim 11, as shown above. Leliwa further discloses, further comprising: requesting the particular intervention to be applied; and "Whenever Online Violence Detection 540 detects any form of online violence, it sends a request for intervention to the following modules of Intervention System 110 along with complete information required for this process" (Leliwa Par. 0109)
storing a log for requesting the particular intervention to be applied separately from a log for applying the particular intervention in the database. "Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message" (Leliwa Par. 0121-0127). "There is also another important feature that can be used to evaluate performed interventions and in turn to provide better interventions in the future. If Online Community 140 utilizes community points or any other form of awarding good contributions, Message Analyzer 114A can proactively request Online Community 140 for such information regarding the intervention message. It can be performed for a predefined period of time in regular intervals. This information can be passed through the following modules of Intervention System 110 and stored in proper databases in order to increase chances of providing good interventions in the future." (Leliwa Par. 0129).
Leliwa discloses requesting the intervention to be applied; applying the intervention; and storing a log for requesting the intervention to be applied separately from a log for applying the intervention in the database. Leliwa, fails to disclose an immutable database. Chen, however, does disclose,
an immutable database “For ease of management of the blockchain, the violation credit record data of the user may be stored into a corresponding blockchain according to the user identifier information of the user. The violation credit record data in the blockchain is stored in the form of a key-value pair, hash operation may be performed according to the user identifier information to obtain a hash value, and hash operation may be performed on a key element in block data to obtain a hash value. If the hash values obtained through two operation processes are consistent, the blockchain is the blockchain corresponding to the user” (Chen Par. 0121-0122). “Blockchain: the blockchain technology is derived from the Bitcoin technology, is an underlying technology of Bitcoin, and is a decentralized distributed ledger database. The blockchain is a string of data blocks (that is, blocks) generated through association by using a cryptographic algorithm, and each data block includes information that is effectively confirmed by a plurality of blockchain network transactions. Based on this, cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured. To clearly describe a blockchain system and a chain formed by blocks in the following description, a blockchain system including a plurality of nodes may be referred to as a blockchain network, and a chain formed by data stored in the blockchain is referred to as a blockchain” (Chen Par. 0033).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of requesting the intervention to be applied; applying the intervention; and storing a log for requesting the intervention to be applied separately from a log for applying the intervention in the database of Leliwa and Chen with an immutable database of Chen because cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured (Chen Par. 0033).
Regarding Claim 7, and Claim 17,
The combination of Leliwa, Chen, and Comartin disclose the method of Claim 1, computer-readable storage medium of claim 11, as shown above. Leliwa further discloses, further comprising: indexing classifications by entities at the database. "Intervention System 110 includes multiple System Databases 112 and multiple System Processors 114 that can be located anywhere that is accessible to a connected network 120, which is typically the Internet. System Databases 112 and System Processors 114 can also be distributed geographically in the known manner. Intervention System 110 uses Online Violence Detection System 130 in order to verify whether or not input text contains online violence and to determine online violence categories" (Leliwa Par. 0023). "Online violence can be broadly defined as any form of abusing, harassing, bullying or exploiting other people, using electronic means. Some communication phenomena such as hate speech, toxic speech or abusive language overlap with online violence to some extent, whereas the other phenomena like cyberbullying or sexual harassment are entirely included into online violence" (Leliwa Par. 0003). "Community Intelligence 114B is a module responsible for analyzing user-related data in order to prepare the most effective intervention. Community Intelligence 114B has access to Community Database 112A, where all user-related data in regard to the given community is stored. The main piece of information stored in Community Database 112A is the whole track record of violent users, including (but not limited to): user identification, timestamp of violence detection, timestamp of sending intervention, type of detected violence (+related words and phrases), type of received intervention, id of received intervention that allows to retrieve an exact text of intervention message" (Leliwa Par. 0121-0127).
Leliwa discloses indexing classifications by entities at the database. Leliwa, fails to disclose an immutable database. Chen, however, does disclose,
at the immutable database “For ease of management of the blockchain, the violation credit record data of the user may be stored into a corresponding blockchain according to the user identifier information of the user. The violation credit record data in the blockchain is stored in the form of a key-value pair, hash operation may be performed according to the user identifier information to obtain a hash value, and hash operation may be performed on a key element in block data to obtain a hash value. If the hash values obtained through two operation processes are consistent, the blockchain is the blockchain corresponding to the user” (Chen Par. 0121-0122). “Blockchain: the blockchain technology is derived from the Bitcoin technology, is an underlying technology of Bitcoin, and is a decentralized distributed ledger database. The blockchain is a string of data blocks (that is, blocks) generated through association by using a cryptographic algorithm, and each data block includes information that is effectively confirmed by a plurality of blockchain network transactions. Based on this, cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured. To clearly describe a blockchain system and a chain formed by blocks in the following description, a blockchain system including a plurality of nodes may be referred to as a blockchain network, and a chain formed by data stored in the blockchain is referred to as a blockchain” (Chen Par. 0033).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of indexing classifications by entities at the database of Leliwa and Chen with an immutable database of Chen because cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured (Chen Par. 0033).
Claim(s) 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Leliwa (US 20200267165 A1), in view of Chen (US20220198468), in view of Comartin (Asynchronous request-response pattern for non-blocking workflows) and in further view of Mu (US 11605017 B1).
Regarding Claim 2, and Claim 12,
The combination of Leliwa, Chen, and Comartin disclose the method of Claim 1, computer-readable storage medium of claim 11, as shown above. The combination of Leliwa, Chen, and Comartin fail to disclose using machine learning or a rule based system to determine new violations. Mu, however, does disclose wherein the determination that there is the new policy violation is determined based on a machine-learning model or a rule-based model. "The online system provides the extracted features to a machine learning based model configured to generate a score indicating whether the content item comprises deceptive information, and executes the machine learning based model to generate a score for the content item. In response to the generated score indicating a likelihood that the content item includes deceptive information, the online system determines whether the content item conforms to content policies of the online system. The online system determines a rate of distribution of the content item to users of the online system based on the determination of whether the content item conforms to content policies of the online system" (Mu Col. 2 lines 13-24).
It would have been obvious to one of ordinary skill in the art to have combined the method of classifying and intervening in online violations of user activity of Leliwa, Chen, and Comartin with using machine learning or a rule based system to determine a new violation of Mu since human reviewers may cause false negatives and if one of the content items was labelled as policy-violating by mistake, then this user's weight generated by the traditional models may decrease dramatically and other bad accounts created by this user may be likely missed by the traditional models (Mu Col 1 Lines 20-40).
Claim(s) 8, 9, 10, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Leliwa (US 20200267165 A1), in view of Chen (US20220198468), in view of Comartin (Asynchronous request-response pattern for non-blocking workflows), and in further view of Square Enix, Frequently Asked Questions, Account Penalty Policy.
Regarding Claim 8, and Claim 18,
The combination of Leliwa, Chen, and Comartin disclose the method of Claim 1, computer-readable storage medium of claim 11, as shown above. Leliwa further discloses, selecting the particular intervention ladder based on the subsequent classification; "It uses responses generated by Intervention System 110 based on type and severity of detected online violence and knowledge about particular violent user and online community" (Leliwa Par. 0043). “The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service.” (Leliwa Par. 0133-0136).
The combination of Leliwa, Chen, and Comartin disclose selecting an intervention ladder based on classifications, and points. The combination of Leliwa, Chen, and Comartin fail to disclose determining a current point value associated with the entity and an associated current level on the intervention ladder based on past policy violation data associated with the previous policy violations, updating the current point value by adding one or more points based on the new policy violation and updating a next level of the intervention ladder wherein each increasing level is associated with a harsher intervention action, wherein the determined intervention is based on the updated point value. Square Enix, Frequently Asked Questions, Account Penalty Policy, however, does disclose,
determining a current point value associated with the entity and an associated current level on the particular intervention ladder based on past policy violation data associated with the previous policy violations; and "Penalty Points refer to specific point values that are assigned to each type of violation category (i.e., "obscene/indecent expressions," "aggressive expressions such as violent language/slander/insults/threats, etc.") and vary depending on the severity of the violation type. The total sum of these Penalty Points determines the type of penalty an offender will receive. For example, if a player receives 10 Penalty Points for "obscene/indecent expressions" and another 5 points for "aggressive expressions such as violent language/slander/insults/threats", then a penalty equivalent to 15 points will be issued on their account. Penalty Points, together with the actual penalty issued, are recorded and will be used to determine the penalties issued for any future violations. All penalties and Penalty Points are recorded and accumulated. As such, in the event a user repeats violations, new Penalty Points will be added, and then a penalty for the new verified violation will be issued based on those points. For example, if a player received 15 points for a previous violation and then committed a new violation worth 10 points, then their latest penalty will be based on the total sum of 25 points, and both the penalty and accrued points will be recorded on their account. It is possible that a Caution or Warning penalty is issued if the previously recorded violation was equivalent to Caution or Warning. However, if the previously recorded violation was a Temporary Service Account Suspension, then a heavier penalty will be imposed even if the new violation is equivalent to a Caution or Warning. For example, if a Temporary Service Account Suspension (3 Days) penalty was issued in the past and a violation equivalent to a Caution is confirmed, a Temporary Service Account Suspension (10 Days) penalty or higher penalty may be issued. However, accrued Penalty Points may be reduced based on the number of days that have passed since the previous penalty" (Square Enix, Frequently Asked Questions, Account Penalty Policy).
updating the current point value by adding one or more points based on the new policy violation and updating a next level of the particular intervention ladder, "All penalties and Penalty Points are recorded and accumulated. As such, in the event a user repeats violations, new Penalty Points will be added, and then a penalty for the new verified violation will be issued based on those points. For example, if a player received 15 points for a previous violation and then committed a new violation worth 10 points, then their latest penalty will be based on the total sum of 25 points, and both the penalty and accrued points will be recorded on their account. It is possible that a Caution or Warning penalty is issued if the previously recorded violation was equivalent to Caution or Warning. However, if the previously recorded violation was a Temporary Service Account Suspension, then a heavier penalty will be imposed even if the new violation is equivalent to a Caution or Warning. For example, if a Temporary Service Account Suspension (3 Days) penalty was issued in the past and a violation equivalent to a Caution is confirmed, a Temporary Service Account Suspension (10 Days) penalty or higher penalty may be issued. However, accrued Penalty Points may be reduced based on the number of days that have passed since the previous penalty" (Square Enix, Frequently Asked Questions, Account Penalty Policy).
wherein each increasing level is associated with a harsher intervention action, wherein the first intervention is based on an updated point value. "There are six levels of penalty in ascending order of severity: "Caution," "Warning," "Temporary Service Account Suspension (3 Days)," "Temporary Service Account Suspension (10 Days)," "Temporary Service Account Suspension (20 Days)," and "Service Account Termination" (Square Enix, Frequently Asked Questions, Account Penalty Policy). "All penalties and Penalty Points are recorded and accumulated. As such, in the event a user repeats violations, new Penalty Points will be added, and then a penalty for the new verified violation will be issued based on those points. For example, if a player received 15 points for a previous violation and then committed a new violation worth 10 points, then their latest penalty will be based on the total sum of 25 points, and both the penalty and accrued points will be recorded on their account. It is possible that a Caution or Warning penalty is issued if the previously recorded violation was equivalent to Caution or Warning. However, if the previously recorded violation was a Temporary Service Account Suspension, then a heavier penalty will be imposed even if the new violation is equivalent to a Caution or Warning. For example, if a Temporary Service Account Suspension (3 Days) penalty was issued in the past and a violation equivalent to a Caution is confirmed, a Temporary Service Account Suspension (10 Days) penalty or higher penalty may be issued" (Square Enix, Frequently Asked Questions, Account Penalty Policy).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the system of detecting online violations of Leliwa, Chen, and Comartin with determining a current point value associated with the entity and an associated current level on the intervention ladder based on past policy violation data associated with the previous policy violations, updating the current point value by adding one or more points based on the new policy violation and updating a next level of the intervention ladder wherein each increasing level is associated with a harsher intervention action, wherein the determined intervention is based on the updated point value of Square Enix, Frequently Asked Questions, Account Penalty Policy since not all actions are as severe, and based on the nature of the violation, an "instruction for improvement" can be provided to the player with guidance on how to improve their behavior to prevent further violations of this type in the future (Square Enix, Frequently Asked Questions, Account Penalty Policy).
Regarding Claim 9, and Claim 19,
The combination of Leliwa, Chen, Comartin and Square Enix, Frequently Asked Questions, Account Penalty Policy, disclose the method of claim 8, computer-readable storage medium of claim 18, as shown above. Chen further discloses further comprising in the immutable database “For ease of management of the blockchain, the violation credit record data of the user may be stored into a corresponding blockchain according to the user identifier information of the user. The violation credit record data in the blockchain is stored in the form of a key-value pair, hash operation may be performed according to the user identifier information to obtain a hash value, and hash operation may be performed on a key element in block data to obtain a hash value. If the hash values obtained through two operation processes are consistent, the blockchain is the blockchain corresponding to the user” (Chen Par. 0121-0122). “Blockchain: the blockchain technology is derived from the Bitcoin technology, is an underlying technology of Bitcoin, and is a decentralized distributed ledger database. The blockchain is a string of data blocks (that is, blocks) generated through association by using a cryptographic algorithm, and each data block includes information that is effectively confirmed by a plurality of blockchain network transactions. Based on this, cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured. To clearly describe a blockchain system and a chain formed by blocks in the following description, a blockchain system including a plurality of nodes may be referred to as a blockchain network, and a chain formed by data stored in the blockchain is referred to as a blockchain” (Chen Par. 0033).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the method of indexing classifications by entities at the database of Leliwa, Chen, Comartin, and Square Enix, Frequently Asked Questions, Account Penalty Policy, with an immutable database of Chen because cheating cannot be performed by tampering with data on the blocks, so that it can be ensured that data on any block is open and the data security is ensured (Chen Par. 0033).
The combination of Leliwa, Chen, and Comartin fail to disclose storing the updated point value and the next level of the intervention ladder in association with the entity. Square Enix, Frequently Asked Questions, Account Penalty Policy, however, further discloses:
storing the updated point value and the next level of the particular intervention ladder in association with the entity "There are six levels of penalty in ascending order of severity: "Caution," "Warning," "Temporary Service Account Suspension (3 Days)," "Temporary Service Account Suspension (10 Days)," "Temporary Service Account Suspension (20 Days)," and "Service Account Termination." (Square Enix, Frequently Asked Questions, Account Penalty Policy). "Penalty Points refer to specific point values that are assigned to each type of violation category (i.e., "obscene/indecent expressions," "aggressive expressions such as violent language/slander/insults/threats, etc.") and vary depending on the severity of the violation type. The total sum of these Penalty Points determines the type of penalty an offender will receive. For example, if a player receives 10 Penalty Points for "obscene/indecent expressions" and another 5 points for "aggressive expressions such as violent language/slander/insults/threats", then a penalty equivalent to 15 points will be issued on their account. Penalty Points, together with the actual penalty issued, are recorded and will be used to determine the penalties issued for any future violations" (Square Enix, Frequently Asked Questions, Account Penalty Policy). "All penalties and Penalty Points are recorded and accumulated. As such, in the event a user repeats violations, new Penalty Points will be added, and then a penalty for the new verified violation will be issued based on those points" (Square Enix, Frequently Asked Questions, Account Penalty Policy).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the system of detecting online violations of Leliwa, Chen, Comartin, and Square Enix, Frequently Asked Questions, Account Penalty Policy, with storing the updated point value and the next level of the intervention ladder in association with the entity of Square Enix, Frequently Asked Questions, Account Penalty Policy, since not all actions are as severe, and based on the nature of the violation, an "instruction for improvement" can be provided to the player with guidance on how to improve their behavior to prevent further violations of this type in the future (Square Enix, Frequently Asked Questions, Account Penalty Policy).
Regarding Claim 10,
The combination of Leliwa, Chen, Comartin, and Square Enix, Frequently Asked Questions, Account Penalty Policy, disclose the method of claim 8, as shown above. Leliwa, further discloses The computer-implemented method of claim 8 further comprising: setting an expiration reset time period for the particular intervention ladder; and "The penalties such as banning are defined by the online community (service). Intervention System 110 can easily adapt to any service and utilize any reasonable combinations of available penalties, including the following aspects: type of penalty: banning, shadow banning, setting restraints on writing/editing; duration: temporary (e.g. 24 hours), permanent; range: selected channel (e.g. thread on forum), whole service" (Leliwa Par. 0133).
Leliwa discloses setting an expiration reset time period for the intervention ladder, but fails to disclose after the expiration reset time period has passed without any intervening policy violations associated with the intervention ladder, resetting the updated point value for the entity. Square Enix, Frequently Asked Questions, Account Penalty Policy, however, does disclose,
after the expiration reset time period has passed without any intervening policy violations associated with the particular intervention ladder, resetting the updated point value for the entity. "If a significant amount of time has passed since the last penalty was issued, then accrued Penalty Points may be reduced based on the content of the previous penalty and the actual number of days that have passed. This does not mean that Penalty Points will suddenly reset to 0 points after a certain number of days. Instead, points will decay in proportion to the number of days elapsed. If the previously issued penalty was either a Caution or Warning, accrued Penalty Points will continue to decay until it reaches 0 points in a one-to-two-year period. However, if the previously issued penalty was a Temporary Service Account Suspension, then the accrued points will not start decaying until a minimum of a three to six-year period has passed and will not reach 0 points for a minimum of seven to ten years. If a new violation is confirmed after the points reduction, the Penalty Points from the new violation will be added to the remaining points from the reduction to determine the penalty to issue for the new violation. The reduction of accrued points is based on the amount of time that has passed since the last penalty. Therefore, if a subsequent violation is committed before a reduction occurs, then the countdown towards a reduction will reset at that point. In addition, this system will not reverse a Service Account Termination" (Square Enix, Frequently Asked Questions, Account Penalty Policy).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the system of detecting online violations of Leliwa, Chen, Comartin, and Square Enix, Frequently Asked Questions, Account Penalty Policy, with after the expiration reset time period has passed without any intervening policy violations associated with the intervention ladder, resetting the updated point value for the entity of Square Enix, Frequently Asked Questions, Account Penalty Policy, since not all actions are as severe, and based on the nature of the violation, an "instruction for improvement" can be provided to the player with guidance on how to improve their behavior to prevent further violations of this type in the future (Square Enix, Frequently Asked Questions, Account Penalty Policy).
Claim(s) 21 are rejected under 35 U.S.C. 103 as being unpatentable over Leliwa (US 20200267165 A1), in view of Chen (US20220198468), and in further view of Comartin (Asynchronous request-response pattern for non-blocking workflows ), and in further view of Li (CN111193774A).
Regarding Claim 21,
The combination of Leliwa, Chen, and Comartin disclose the method of Claim 1, as shown above. The combination of Leliwa, Chen, and Comartin fail to disclose the detection service being a scalable microservice configured to operate asynchronously and in parallel such that policy violations are processed with low latency. Li, however, discloses processing messages. Li further discloses wherein the detection service is a scalable microservice configured to operate asynchronously and in parallel such that policy violations are processed with low latency. "The result is synchronously given, the strategy of asynchronous thread opening for processing can not only enable the request to be quickly responded and improve the throughput of the system, but also can improve the processing speed of concurrent access of mass data by asynchronously thread opening for concurrently processing each piece of data, and the asynchronization is realized in a mode of Dubbo scheduling and asynchronous kafka message queue notification in an actual project. In addition to the above, the present embodiment also adopts a distributed microservice architecture to improve data processing speed and improve resource utilization. Distributed computing is a study on how to divide a problem that needs huge computing power to solve into many small parts, then distribute the parts to many computers for processing, and finally combine the computing results to obtain the final result. The distributed computing and the parallel computing have different pursued effects, the distributed computing is concerned with reliability, and the parallel computing is concerned with speed, so that the combination of the distributed computing and the parallel computing can improve the speed of computing response of the system and ensure the reliability of a computing result" (Li Par. 0158-0160).
It would have been obvious to one of ordinary skill in the art to have combined the method of classifying and intervening in online violations of user activity of Leliwa, Chen, and Comartin with a scalable microservice configured to operate asynchronously and in parallel such that policy violations are processed with low latency of Li to improve data processing speed and improve resource utilization (Li Par. 0159).
Response to Arguments
Applicant's arguments filed 03/13/2026 with respect to 35 U.S.C. § 101, have been fully considered but they are not persuasive. Applicant argues that under Step 2A Prong One that the claims are not “directed to” a judicial exception because the claim is not merely reciting what moderation decision is made, but recites how a particular distributed computer system is configured to process high-volume near-real-time communications without degrading responsiveness, and that the claim requires “receiving, by a safety gateway, incoming data from the entity that is performing an action at the server, wherein the safety gateway is configured to asynchronously queue and forward the incoming data to a detection service without blocking or serializing the action performed by the entity” which is not something a person “with or without the aid of a computer” can practically perform. The Examiner respectfully disagrees. As noted above, other than general recitation of an immutable database, a server, a safety gateway, a non-transitory computer-readable storage medium, a computer, a processor, and memory (generic computer elements), the concepts of reading a history of past violations, receiving incoming data without blocking or serializing an action, determining a new policy violation associated with a subsequent classification that indicated a type of policy violation and determined on the previous classifications, selection of a particular intervention ladder, determination of an intervention for the new policy violation, application of the intervention, and storing the new policy violation, falls withing certain methods of organizing human activities. As recited, the claims do not recite any specific improvement to computer technology. The claims simply utilize conventional computing components to read, receive, and determine violations while selecting, determining, and applying interventions. As such the claim recites an abstract idea within the group of certain methods of organizing human activity.
The Applicant further argues under Step 2A, Prong Two that even if an abstract idea is identified, the claim integrates it into a practical application by explicitly preventing the moderation pipeline from blocking or serializing the underlying server action by “a server that supports near real-time communications” together with “a safety gateway… configured to asynchronously queue and forward… without blocking or serializing the action”, together with “applying the first intervention by an intervention service, wherein the intervention service uses a state of the entity’s intervention in applying the first intervention”, together with “reading a state of an immutable database…” and “storing… in the immutable database” which improves system responsiveness. The Examiner respectfully disagrees. The claims as recited do not recite any specific technical improvement to the additional elements of a computer, server, safety gateway, immutable database, processor, or memory. The additional elements are merely additional elements representing conventional computer technologies employed to apply the underlying abstract idea. Here, the Applicant has not identified nor can the Examiner locate any physical improvement to the functioning of the generic computing components that result from the implantation of Applicant’s claim. The Examiner reinstates that fact that the action occurs “in real time” is simply a matter of when the analysis occurs, not how it improves computer technology. At best, Applicant’s identified problem is a business/administrative problem. Because no technological problem is present, the claims do not integrate the abstract idea into a practical application.
The Applicant further argues under Step 2B that the additional elements are not “mere instructions to apply” an exception. Applicant argues that the claims requires a server “that supports near real-time communications”, a safety gateway configured to "asynchronously queue and forward" incoming data "without blocking or serializing the action", an intervention service that "uses a state of the entity's intervention," and an immutable database whose "state" is read and then appended with the resulting violation/classification/intervention, and recite an inventive distribution of functionality within a network (e.g., content filtering), and “the detection service is a scalable microservice configured to operate asynchronously and in parallel such that policy violations are processed with low latency”. The Examiner respectfully disagrees, and the arguments above also apply to these arguments. The use of a server for near real-time communication, a safety gateway to queue and forward data, an intervention service that uses a state of the entity’s intervention, immutable database are merely claimed broadly such that they are no more than using computer programs in their ordinary capacity to determine and apply interventions. Furthermore, even when considering the intervention system is performed by a sever, safety gateway, and immutable database, this is still not an improvement to computer functionality, because it is equivalent to “apply it” or mere instructions to perform the abstract idea on a generic computer. Even viewing the claims as a whole, including all of the steps allegedly included in the applicant’s remarks, the claims do not provide significantly more than the abstract idea. Therefore, the rejection of claimed 35 U.S.C. 101 stands.
Applicant's arguments filed 03/10/2026 with respect to 35 U.S.C. § 103, have been fully considered, but they are not persuasive. Applicant argues that the cited references fail to teach the amendments. The rejection has been updated in light of the amendments, therefore, making the arguments moot. The 103 Rejection is maintained.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emily M Kraisinger whose telephone number is (703)756-4583. The examiner can normally be reached M-F 7:30 AM -4:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jessica Lemieux can be reached at 571-270-3445. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.M.K./Examiner, Art Unit 3626
/JESSICA LEMIEUX/Supervisory Patent Examiner, Art Unit 3626
Prosecution Insights