Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responding to application papers dated 5/16/2023.
Claims 1-20 are pending in the application.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Specifically, claims 1-20 are directed to an abstract idea.
Per claim 1, the claim is directed to an idea of itself, mental processes that can be performed in the human mind, or by a human using a pen and paper. The steps of assigning, detecting a conflict and generating as drafted can be pure mental processes. Assigning an experiment, detecting a conflict based on the assignments and generating an indication of the conflict can be mentally designed mentally in a design phase. The displaying of the indication in a user interface is in fact an intended action only (Even assuming that the display is the actual action, it is a mere display of the result which is an insignificant extra solution activity; however, “for display” is considered as an intended action). The additional limitations, the step of receiving an indication and request are mere data gathering for the mental steps of detecting and generation while the processing device is described at a high level of generality for applying or performing the abstract idea and do not indicate any integration of the abstract idea into a practical application as the mental steps are merely applied with a generic computing component(s). See MPEP see MPEP 2106.05(f) /2106.05(h). It is noted that employing generic computer functions to execute an abstract idea, even when limiting the use of the idea to one particular environment, does not add significantly more, similar to how limiting the abstract idea in Flook to petrochemical and oil-refining industries was insufficient. Therefore, the additional limitations do not integrate the abstract idea into a practical application. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the recitation of generic computer components or insignificant extra solution activities (e.g. processors, devices, program instructions), then it falls within the "Mental Processes" grouping of abstract ideas (2019 PEG step 2A, Prong 1: Abstract idea grouping? Yes, Mental Process). At most, the receiving step is not found to include anything more than what is well-understood, routine, conventional activity in the field. In this case, it is noted that the claimed extra-solution of data gathering is acknowledged to be a well-understood, routine, conventional activity court recognized as WURC examples in MPEP 2106.05(d)(ll), for example, data gathering and retrieving, storing data, transmitting/displaying a result - Symantec, Versata Dev, Content extraction, Electric Power Group). Insignificant extra solution activities or mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Viewing the limitations individually and as a combination, the additional elements merely perform data gathering for the mental steps and perform the mental steps using generic computing components as tools without integrating the abstract idea into a practical application. For at least these reasons, claim 1 is not patent eligible.
Per claims 2-10, these claims are directed to the same idea itself as in claim 1, reciting details of the mental steps without adding any other additional element that is significantly more. Per claims 2, 3, 5-10, these claims further recite the mental steps, and generating a report summarizing results from mental models can be done mentally by entity association. Per claim 4, performing the experiments in parallel is recited as a concept that resolves the conflict, not actual performances in parallel. Therefore, the claims are rejected for the same reasons as in claim 1.
Per claim 11, the claim is directed to an idea of itself, mental processes that can be performed in the human mind, or by a human using a pen and paper. The steps of performing the first and second experiments, detecting a conflict and resolving the conflict as drafted can be pure mental processes. The additional limitations, the step of receiving a request is mere data gathering for the mental steps while the memory component, and processing device are described at a high level of generality for applying or performing the abstract idea and do not indicate any integration of the abstract idea into a practical application as the mental steps are merely applied with a generic computing component(s). See MPEP see MPEP 2106.05(f) /2106.05(h). It is noted that employing generic computer functions to execute an abstract idea, even when limiting the use of the idea to one particular environment, does not add significantly more, similar to how limiting the abstract idea in Flook to petrochemical and oil-refining industries was insufficient. Therefore, the additional limitations do not integrate the abstract idea into a practical application. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the recitation of generic computer components or insignificant extra solution activities (e.g. processors, devices, program instructions), then it falls within the "Mental Processes" grouping of abstract ideas (2019 PEG step 2A, Prong 1: Abstract idea grouping? Yes, Mental Process). At most, the receiving step is not found to include anything more than what is well-understood, routine, conventional activity in the field. In this case, it is noted that the claimed extra-solution of data gathering is acknowledged to be a well-understood, routine, conventional activity court recognized as WURC examples in MPEP 2106.05(d)(ll), for example, data gathering and retrieving, storing data, transmitting/displaying a result - Symantec, Versata Dev, Content extraction, Electric Power Group). Insignificant extra solution activities or mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Viewing the limitations individually and as a combination, the additional elements merely perform data gathering for the mental steps and perform the mental steps using generic computing components as tools without integrating the abstract idea into a practical application. For at least these reasons, claim 11 is not patent eligible.
Per claims 12-15, these claims are directed to the same idea itself as in claim 11, reciting details of the mental steps without adding any other additional element that is significantly more. The step of generating a report summarizing results from mental models can be done mentally by entity association. Therefore, the claims are rejected for the same reasons as in claim 11.
Per claims 16-20, these claims are directed to the same idea itself as in claim 1-10 reciting the same mental steps and the medium recited at the preamble which is a generic computing component without adding any other additional element that is significantly more. Therefore, the claims are rejected for the same reasons as in claim 11.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-8, 10, 11, and 14-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mordo et al. (US20160124839, hereafter Mordo).
1. A method comprising: assigning, by a processing device, a first experiment to a first namespace and a second experiment to a second namespace (Mordo, see at least [0175] experiments E1 and E2 both modify the same website building system objects components;[0264] by limiting experiments to these whose scope is that of the specific sub-system used (e.g. handle only experiments related to the system editor); [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based (for example) on a combination of any of the following: a priority assigned by the developer, a priority assigned by the experiment manager, the experiment scope (i.e. affected system area)—assuming some system areas are more important than others; [0188] the variant selection for both experiments may be done according to the leading experiment's ID; [Note that limiting experiments to a specific scope is a form of namespace, where logical grouping (scope) for experiments prevents conflicts and manages access, creating isolated environments. Limiting experiments with unique IDs to the specific context, like the system editor creates a distinct scope for the experiments isolating experiment ID within a defined boundary where the editor experiments live within their space separate from the core system’s entities. An experiment is limited to its own experiment scope (namespace)].
receiving, by the processing device via a network, an indication of the first experiment performed via a first channel using a first set of entities and a second set of entities from a group of entities (Mordo, see at least [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics … A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [0186]; [0221] a user might ask to participate in a particular experiment instead of being assigned one randomly (e.g. an advanced user which would like to focus on a new, not fully released feature); [0257] once experiments are running, … terminating an experiment, discarding an experiment… when merging (for example) experiments E1 and E2, a specific user might have a A-variant selection in experiment E1 and a B-variant selection in experiment E2; [0277] Experiment manager 244 may accept the request and experiment configurer 243 may test if the user is a part of a user group for which a user group state-map is specified; [0310]; [0381] If a specific experiment E has proved to be a success, it would be added (eventually) to all users; [Note that users are segmented into different sets of entities and each experiments are performed via different channels using particular groups of users]);
receiving, by the processing device via the network, a request to perform the second experiment via a second channel using a third set of entities and a fourth set of entities from the group of entities (Mordo, see at least [0248] Each user invoking system 100 may receive a different version of the system based on explicit invocation of a specific state map; [0251] Users may receive the B-variant of a component (different from the deployed A-variant) due to AB-testing of an experiment involving the specific component;[0267] a user may enter system 100 and may request a copy of the system client code 50 to run on his user machine 10 from system server set 200; [0442] System 100 may report the final outcome to the person requesting the modification; [0056] support developer and user supplied requests for how an experiment should be run and the variants used. …a user to request to participate in a particular experiment; [0265] determine which experiments to configure for a given user based on user group state maps; [0277] Experiment manager 244 may accept the request and experiment configurer 243 may test if the user is a part of a user group for which a user group state-map is specified. If so experiment coordinator 330 may activate the state-map and set of experiments accordingly; [0381] If a specific experiment E has proved to be a success, it would be added (eventually) to all users, including users in the “A group” who didn't see it before; [0133] Experiment running (430) (performed by experiment manager 244)—running the system with active experiments (which are applied to entering users) while it collects the results. …when a given experiment reached a pre-specified number Y of users (which received variant B instead of the regular B); [Note that users are segmented into different sets of entities and experiments can be performed via different channels using particular groups of users])).
detecting, by the processing device, a conflict for the second experiment based on the first namespace and the second namespace (Mordo, see at least [0016] a conflict resolver to detect and resolve conflicts between the experiment and at least one of: the least one other experiment and other system aspects; [0036] the detecting and resolving resolves conflicts between the experiment and at least one other experiment using experiment separation; [0057]; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122] [0198]; [0199]; [0285] active conflict resolver 370 may detect any conflicts in the calculated experiment assignment (not resolved before) and break each conflict by either using experiment separation so to run each of the experiment on different population (e.g. based on ID) as detailed below or by dropping one of the experiments (with the lower experiment priority); [Note that a conflict is detected for experiments with a specific scope upon integration of one experiment into the system that it conflicts with an existing experiment]).
generating, by the processing device, an indication of the conflict for the second experiment for display in a user interface (Mordo, see at least [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122] experiment managers may work with the UI server which would use an API provided by an experiment handling server. Thus experiment server 240 may also serve as a communication interface allowing developers users etc. to input specifications, state maps, modifications etc., but also to receive warnings, BI reports etc. in order to promote decision making; [0054] Applicants have realized that an experiment management system is desirable for managing multiple experiments inside a component based interactive system such as a website building system. Applicants have further realized that the systems of the prior art are limited in the type of tests they can support and cannot handle changes which may affect both code and web pages or a combination of the two; [0100]; [0247] Conflict resolver 248 may also be used to detect conflicts and dependencies in such merged state maps, warn about them, and resolve them … interactively; [Note that reporting a result or information to a user indicates generating an indication of that result for display, to communicate the outcome clearly and effectively to the user and interactive conflict resolution involves with displaying the conflicting data to the user]).
2. The method as described in claim 1, wherein assigning the second experiment to the first namespace resolves the conflict for the second experiment (Mordo, see at least [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based (for example) on a combination of any of the following: a priority assigned by the developer, a priority assigned by the experiment manager, the experiment scope (i.e. affected system area); [0281] Active conflict resolver 370 may also detect any dependency circles in the calculated experiment assignment (not resolved before) and break them by dropping one of experiments so to break the cycle; [0285] active conflict resolver 370 may detect any conflicts in the calculated experiment assignment (not resolved before) and break each conflict by either using experiment separation so to run each of the experiment on different population (e.g. based on ID) as detailed below or by dropping one of the experiments (with the lower experiment priority); [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution and experiment implementer 320 may then send this version to client 10 for execution; [Note that conflicts detected for experiments are resolved by experiments assignment] by the conflict resolver).
3. The method as described in claim 2, wherein the first experiment evaluates a first value of a variable and the second experiment evaluates a second value of the variable (Mordo, see at least [0324] [0064]; [0061]; [0063]; [0455] coordinate multiple different experiments under a single unified framework with the ability to handle both code and web pages. System 100 may simultaneously set up and run different types of experiments such as A/B tests, multivariate tests, marketing campaigns, feature toggles and system updates allowing for the collection and analysis of the results of the experiments and modifications to the running of the experiments as a result; [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics (such as session length, conversion to paid users etc.). A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [Note that a single variable is experimented to take on different values across separate experiments comparing multiple variants of a single variable]).
4. The method as described in claim 1, wherein performing the second experiment in parallel with performing the first experiment resolves the conflict for the second experiment (Mordo, see at least [0057] detect and resolve conflicts between different experiments that are running concurrently as well as have the ability to update the parameters of the experiments and baselines according to an analysis of the results; [0351] running conflicting experiments simultaneously but separately … the failed experiments are discard and the conflict is resolved. This may save coordination work, and may make parallel development easier; [0390] multi-tenancy support, allowing multiple distinct user classes to use the same experiment management system infrastructure and conduct experiments in parallel; [Note conducting experiments in parallel resolve conflicts for certain experiment]).
5. The method as described in claim 1, further comprising identifying a particular entity included in the first set of entities and the third set of entities ( [0099] The system has performed a secondary conflict check (as described below) and it did not result in canceling the experiment X for the given user, e.g. due to conflict with another (higher priority) experiments for the same user; [0266] experiment selector 310 may select an experiment set for a particular user based on the entered population selection criteria; [0272] Experiment analyzer 251 may later correlate these multiple ID to create merged information reports (e.g. merged BI records under the anonymous used ID and the regular user ID of the same user; [0277] test if the user is a part of a user group for which a user group state-map is specified; [0287] Variant modifier 380 may perform variant selection for all experiments open for the particular user which do not have an existing variant selection (as described in more detail herein below); [Note that a particular user can be identified and selected based on the entered population selection criteria and user group state-map]).
6. The method as described in claim 5, wherein the particular entity is associated with a first identification for the first channel and a second identification for the second channel (Mordo, see at least [0270] multiple ID's (of different types) for the same user ... a specialized anonymous user ID for repeating anonymous users (e.g. tracking them through an inserted cookie). The regular user ID would only be assigned later when the user actually registers or logs into the system; [0272] Experiment analyzer 251 may later correlate these multiple ID to create merged information reports (e.g. merged BI records under the anonymous used ID and the regular user ID of the same user); Note that a user is associated with user IDs for different channels which are correlated (stitched) later for identify resolution)).
7. The method as described in claim 6, wherein the first identification and the second identification are associated with the particular entity using identity stitching (Mordo, see at least [0272] Experiment analyzer 251 may later correlate these multiple ID to create merged information reports (e.g. merged BI records under the anonymous used ID and the regular user ID of the same user); Note that correlating multiple IDs for the same user creates a single, merged profile that is precisely what identity stitching (identity resolution) is).
8. The method as described in claim 5, wherein the third set of entities and the fourth set of entities are formed by segmenting the first set of entities into the third set of entities and the fourth set of entities (Mordo, see at least [0309] Another type of path expansion may involve registered users only. Population expander 247 may split these users “new users” (registered after the experiment begun) and non-new users. Population expander 247 may allow expansion from new users to all registered users; [0310] Population expander 247 may split the users into “1st time anonymous users” and “repeating anonymous users” (detected according to a cookie installed on their machine). Population expander 247 may allow expansion from “1st time anonymous users” to “all anonymous users”; [0350] Active conflict resolver 370 may also divide the relevant user population between the conflicting experiments, e.g. if experiment E1 and E2 should each be tested on 10% of a relevant user population, each would be allocated a separate 10% “block”. Such separation is performed at the running stage; [0381]; [0414] use user base segmentation information; [Note that users can be segmented into different sets of entities]).
10. The method as described in claim 9, wherein the conflict for the second experiment is resolved by assigning the second experiment to the first namespace (Mordo, see at least [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based … the experiment scope (i.e. affected system area)—assuming some system areas are more important than others, the running time of the experiment and how close the experiment is to termination (based on achieving a statistically significant result); [0281]; [0285] It will be appreciated that active conflict resolver 370 may detect any conflicts in the calculated experiment assignment (not resolved before) and break each conflict by either using experiment separation so to run each of the experiment on different population (e.g. based on ID) as detailed below or by dropping one of the experiments (with the lower experiment priority; [Note that conflicts detected for experiments are resolved by experiments assignment]).
11. A system comprising: a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising: performing a first experiment via a first channel to evaluate first values of a variable, the first experiment assigned to a first namespace (Mordo, see at least [0175] experiments E1 and E2 both modify the same website building system objects components;[0264] by limiting experiments to these whose scope is that of the specific sub-system used (e.g. handle only experiments related to the system editor); [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based (for example) on a combination of any of the following: a priority assigned by the developer, a priority assigned by the experiment manager, the experiment scope (i.e. affected system area)—assuming some system areas are more important than others; [0188] the variant selection for both experiments may be done according to the leading experiment's ID; [Note that limiting experiments to a specific scope is a form of namespace, where logical grouping (scope) for experiments prevents conflicts and manages access, creating isolated environments. Limiting experiments with unique IDs to the specific context, like the system editor creates a distinct scope for the experiments isolating experiment ID within a defined boundary where the editor experiments live within their own space separate from the core system’s entities. An experiment is limited to its own experiment scope (namespace)].
receiving, via a network, a request to perform a second experiment via a second channel to evaluate second values of the variable, the second experiment assigned to a second namespace (Mordo, see at least [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics … A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [0186]; [0221] a user might ask to participate in a particular experiment instead of being assigned one randomly (e.g. an advanced user which would like to focus on a new, not fully released feature); [0257] once experiments are running, … terminating an experiment, discarding an experiment… when merging (for example) experiments E1 and E2, a specific user might have a A-variant selection in experiment E1 and a B-variant selection in experiment E2; [0277]; [0310]; [0381]; [0251] Users may receive the B-variant of a component (different from the deployed A-variant) due to AB-testing of an experiment involving the specific component;[0267]; [0442]; [0056] support developer and user supplied requests for how an experiment should be run and the variants used. …a user to request to participate in a particular experiment; [0265] determine which experiments to configure for a given user based on user group state maps; [0133]; [Note that users are segmented into different sets of entities and experiments can be performed via different channels using particular groups of users]);
detecting a conflict for the second experiment based on the first namespace and the second namespace; (Mordo, see at least [0016] a conflict resolver to detect and resolve conflicts between the experiment and at least one of: the least one other experiment and other system aspects; [0036] the detecting and resolving resolves conflicts between the experiment and at least one other experiment using experiment separation; [0057]; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122] [0198]; [0199]; [0285] active conflict resolver 370 may detect any conflicts in the calculated experiment assignment (not resolved before) and break each conflict by either using experiment separation so to run each of the experiment on different population (e.g. based on ID) as detailed below or by dropping one of the experiments (with the lower experiment priority); [Note that a conflict is detected for experiments with a specific scope upon integration of one experiment into the system that it conflicts with an existing experiment]).
resolving the conflict for the second experiment by assigning the second experiment to the first namespace; and performing the second experiment via the second channel to evaluate the second values of the variable (Mordo, see at least [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122] experiment managers may work with the UI server which would use an API provided by an experiment handling server. Thus experiment server 240 may also serve as a communication interface allowing developers users etc. to input specifications, state maps, modifications etc., but also to receive warnings, BI reports etc. in order to promote decision making; [0054] Applicants have realized that an experiment management system is desirable for managing multiple experiments inside a component based interactive system such as a website building system. Applicants have further realized that the systems of the prior art are limited in the type of tests they can support and cannot handle changes which may affect both code and web pages or a combination of the two; [0100]; [0247] Conflict resolver 248 may also be used to detect conflicts and dependencies in such merged state maps, warn about them, and resolve them … interactively; [0324] [0064]; [0061]; [0063]; [0455] coordinate multiple different experiments under a single unified framework with the ability to handle both code and web pages. System 100 may simultaneously set up and run different types of experiments such as A/B tests, multivariate tests, marketing campaigns, feature toggles and system updates allowing for the collection and analysis of the results of the experiments and modifications to the running of the experiments as a result; [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics (such as session length, conversion to paid users etc.). A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [Note that a single variable is experimented to take on different values across separate experiments comparing multiple variants of a single variable]).
14. The system as described in claim 11, wherein the operations further comprise: receiving a request to perform a third experiment using a set of entities used to perform the first experiment; and performing the third experiment using the set of entities used to perform the first experiment (Mordo, see at least [0248] Each user invoking system 100 may receive a different version of the system based on explicit invocation of a specific state map; [0251] Users may receive the B-variant of a component (different from the deployed A-variant) due to AB-testing of an experiment involving the specific component;[0267] a user may enter system 100 and may request a copy of the system client code 50 to run on his user machine 10 from system server set 200; [0056] support developer and user supplied requests for how an experiment should be run and the variants used. …a user to request to participate in a particular experiment; [0265] determine which experiments to configure for a given user based on user group state maps; [0277] test if the user is a part of a user group for which a user group state-map is specified. If so experiment coordinator 330 may activate the state-map and set of experiments accordingly; [0381] If a specific experiment E has proved to be a success, it would be added (eventually) to all users, including users in the “A group” who didn't see it before; [0133] (which received variant B instead of the regular B); [0099]; [0266] experiment selector 310 may select an experiment set for a particular user based on the entered population selection criteria; [0272] Experiment analyzer 251 may later correlate these multiple ID to create merged information reports (e.g. merged BI records under the anonymous used ID and the regular user ID of the same user; [0277] test if the user is a part of a user group for which a user group state-map is specified; [0287] Variant modifier 380 may perform variant selection for all experiments open for the particular user which do not have an existing variant selection (as described in more detail herein below); [Note that a particular user can be identified and selected based on the entered population selection criteria and user group state-map]).
15. The system as described in claim 14, wherein the third experiment is assigned to the second namespace (Mordo, see at least [0248] Each user invoking system 100 may receive a different version of the system based on explicit invocation of a specific state map; [0251] Users may receive the B-variant of a component (different from the deployed A-variant) due to AB-testing of an experiment involving the specific component;[0267] a user may enter system 100 and may request a copy of the system client code 50 to run on his user machine 10 from system server set 200; [0442] System 100 may report the final outcome to the person requesting the modification; [0056] support developer and user supplied requests for how an experiment should be run and the variants used. …a user to request to participate in a particular experiment; [0265] determine which experiments to configure for a given user based on user group state maps; [0277] Experiment manager 244 may accept the request and experiment configurer 243 may test if the user is a part of a user group for which a user group state-map is specified. If so experiment coordinator 330 may activate the state-map and set of experiments accordingly; [0381] If a specific experiment E has proved to be a success, it would be added (eventually) to all users, including users in the “A group” who didn't see it before; [0133] Experiment running (430) (performed by experiment manager 244)—running the system with active experiments (which are applied to entering users) while it collects the results. …when a given experiment reached a pre-specified number Y of users (which received variant B instead of the regular B)). [Note that experiments can be assigned to different namespaces(scopes)]).
Per claims 16-20, they are the medium versions of claims 1-7, respectively, and are rejected for the same reasons set forth in connection with the rejection of claims 1-7 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 9, 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Moordo in view of Padala et al. (US20220276953, hereafter padala).
Per claim 9:
Mordo teaches resolving the conflict for the second experiment; and generating a report summarizing results of the first experiment by associating a particular entity from the group of entities with a first identification for the first channel and a second identification for the second channel (Mordo, see at least [0280] conflict resolver 248 which may resolve dependency circles and conflicts at the experiment integration or setup stages through interaction with the developer/integrator, active conflict resolver 370 may resolve all remaining conflicts and issues automatically. Active conflict resolver 370 may resolve based on experiment priority calculated … resolve based number of users affected by this experiment; [0281]; [0285]; [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution and experiment implementer 320 may then send this version to client 10 for execution).
Mordo does not explicitly teach the report includes a summary of the results. However
Padala teaches providing a summary (Padala, see at least [0033] When the defined termination condition is satisfied, the test code executing in container instances 138 may terminate testing and generate a report summarizing the results of the tests executed against compiled application source code in application namespace 132). It would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to have combined Padala’s summary with Mordo’s testing system to modify Mordo’s system to combine the summary function as taught by Padala, with a reasonable expectation of success, since they are analogous art because they are from the same field of endeavor related to testing. Combining Padala’s functionality with that of Mordo results in a system that provides a summary of the experiments. The modification would be obvious because one having ordinary skill in the art would be motivated to make this combination to present a user a report summarizing the results of the experiments for convenience (Padala, see at least [0033] When the defined termination condition is satisfied, the test code executing in container instances 138 may terminate testing and generate a report summarizing the results of the tests executed against compiled application source code in application namespace 132).
Per claims 12 and 13, they are the system versions of claims 6 and 9, respectively, and are rejected for the same reasons set forth in connection with the rejection of claims 6 and 9 above.
Examiner’s Note
The Examiner has pointed out particular references contained in the prior art of record within the body of this action for the convenience of the Applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply. Applicant, in preparing the response, should consider fully the entire reference as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US10951503 is related to determining validity of data obtained from an A/B experiment;
CN114741124 is related to experiment efficiency in the AB experiment process;
US20220413991 is related to dividing a group of user devices into a first subset and a second subset of user devices to participate in a controlled build rollout of a second version of the software product;
US20230273874 is related to imbalance detection in testing;
US20170316123 is related to bias removing in testing;
US 20200272548 is related to different functional verification tests to be divided into namespaces.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to INSUN KANG whose telephone number is (571)272-3724. The examiner can normally be reached M-TR 8 -5pm; week 2: Tu-F 8-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chat Do can be reached at 571-272-3721. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/INSUN KANG/Primary Examiner, Art Unit 2193