Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responding to the amendment filed on 4/14/2026.
Claims 1, 3-9, 11-16, and 18-23 are pending in the application.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim 1-8, 10, 11, and 14-22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mordo et al. (US20160124839, hereafter Mordo).
Per claim 1:
Mordo teaches: A method comprising: assigning, by a processing device, a first experiment defining a mutually exclusive set of entities and a second experiment to a second namespace defining a mutually exclusive set of entities (Mordo, see at least [0175] experiments E1 and E2 both modify the same website building system objects components;[0264] by limiting experiments to these whose scope is that of the specific sub-system used (e.g. handle only experiments related to the system editor); [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based (for example) on a combination of any of the following: a priority assigned by the developer, a priority assigned by the experiment manager, the experiment scope (i.e. affected system area)—assuming some system areas are more important than others; [0188] the variant selection for both experiments may be done according to the leading experiment's ID; [0181], [0122]; [0258]; [0181] Experiment A changes a given button to the color red is mutually exclusive (<ME> below) with experiment B which changes the background of this button to the color red. Such exclusion would be required even if the button drawing and background drawing are handled by separate and independent parts of the system; TABLE-US-00001 Category Code Relationship Description Inclusion/ <MI> Must include If [A <MI> B], than A may only be applied exclusion if B is applied as well - either before or after A; [Note that limiting experiments to a specific scope is a form of namespace, where logical grouping (scope) for experiments prevents conflicts and manages access, creating isolated environments. Limiting experiments with unique IDs to the specific context, like the system editor creates a distinct scope for the experiments isolating experiment ID within a defined boundary where the editor experiments live within their space separate from the core system’s entities. An experiment is limited to its own experiment scope (namespace). The system flags the experiment A and B as mutually exclusive because letting both run together would product conflicting result and the active conflict resolver 370 resolves <ME> conflicts using experiment separation by splitting or dropping one of the two experiments].
receiving, by the processing device via a network, an indication that the first experiment is to be performed via a first channel using a first set of entities and a second set of entities from a group of entities (Mordo, see at least [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics … A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [0186]; [0221] a user might ask to participate in a particular experiment instead of being assigned one randomly (e.g. an advanced user which would like to focus on a new, not fully released feature); [0257] once experiments are running, … terminating an experiment, discarding an experiment… when merging (for example) experiments E1 and E2, a specific user might have a A-variant selection in experiment E1 and a B-variant selection in experiment E2; [0277] Experiment manager 244 may accept the request and experiment configurer 243 may test if the user is a part of a user group for which a user group state-map is specified; [0310]; [0381] If a specific experiment E has proved to be a success, it would be added (eventually) to all users; [Note that users are segmented into different sets of entities and each experiments are performed via different channels using particular groups of users]);
receiving, by the processing device via the network, a request to perform the second experiment via a second channel using a third set of entities and a fourth set of entities from the group of entities (Mordo, see at least [0248] Each user invoking system 100 may receive a different version of the system based on explicit invocation of a specific state map; [0251] Users may receive the B-variant of a component (different from the deployed A-variant) due to AB-testing of an experiment involving the specific component;[0267] a user may enter system 100 and may request a copy of the system client code 50 to run on his user machine 10 from system server set 200; [0442] System 100 may report the final outcome to the person requesting the modification; [0056] support developer and user supplied requests for how an experiment should be run and the variants used. …a user to request to participate in a particular experiment; [0265] determine which experiments to configure for a given user based on user group state maps; [0277] Experiment manager 244 may accept the request and experiment configurer 243 may test if the user is a part of a user group for which a user group state-map is specified. If so experiment coordinator 330 may activate the state-map and set of experiments accordingly; [0381] If a specific experiment E has proved to be a success, it would be added (eventually) to all users, including users in the “A group” who didn't see it before; [0133] Experiment running (430) (performed by experiment manager 244)—running the system with active experiments (which are applied to entering users) while it collects the results. …when a given experiment reached a pre-specified number Y of users (which received variant B instead of the regular B); [Note that users are segmented into different sets of entities and experiments can be performed via different channels using particular groups of users])).
detecting, by the processing device, a conflict for the second experiment based on the first namespace and the second namespace as evaluating a value of a same variable (Mordo, see at least [0032] Moreover, in accordance with a preferred embodiment of the present invention, the experiment is at least one of: an AB test experiment; [0036] the detecting and resolving resolves conflicts between the experiment and at least one other experiment using experiment separation; [0057]; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122]; [0198]; [0199]; [0285] active conflict resolver 370 may detect any conflicts in the calculated experiment assignment (not resolved before) and break each conflict by either using experiment separation so to run each of the experiment on different population (e.g. based on ID) as detailed below or by dropping one of the experiments (with the lower experiment priority); [0174] Conflict resolver 248 may also check for ambiguity and/or collisions. …. When multiple experiments modify different methods of the same module, system 100 may be configured to classify this as collision or not (i.e. configured for different collision checking granularity); [0175] experiments E1 and E2 both modify the same website building system objects components (and in particular the same attribute) and therefore may also create a problem case. This may be extended to contained objects (e.g. experiment E1 modifies the color of the component X and experiment E2 hides the container containing X); [0176]; [0181]; [0217]; [0236]; [0410]; [0091] A/B tests may support user experience consistency where the same variant selection is used whenever the user re-enters the system; Note that a variable refers to an element on a website or application that can have multiple states such as font size, color, button action etc. and the collision/ambiguity happens when wo experiments try to modify or evaluate the exact same variable for the same user because both experiments are trying to assign a value to the same variable, such as in the A/B test.
resolving, by the processing device, the conflict by assigning the second experiment to the first namespace (Mordo, see at least [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based (for example) on a combination of any of the following: a priority assigned by the developer, a priority assigned by the experiment manager, the experiment scope (i.e. affected system area); [0281] Active conflict resolver 370 may also detect any dependency circles in the calculated experiment assignment (not resolved before) and break them by dropping one of experiments so to break the cycle; [0285] active conflict resolver 370 may detect any conflicts in the calculated experiment assignment (not resolved before) and break each conflict by either using experiment separation so to run each of the experiment on different population (e.g. based on ID) as detailed below or by dropping one of the experiments (with the lower experiment priority); [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution and experiment implementer 320 may then send this version to client 10 for execution; [Note that conflicts detected for experiments are resolved by experiments assignment by the conflict resolver).
performing, by the processing device, the second experiment in parallel with the first experiment (Mordo, see at least [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122] experiment managers may work with the UI server which would use an API provided by an experiment handling server. Thus experiment server 240 may also serve as a communication interface allowing developers users etc. to input specifications, state maps, modifications etc., but also to receive warnings, BI reports etc. in order to promote decision making; [0054] Applicants have realized that an experiment management system is desirable for managing multiple experiments inside a component based interactive system such as a website building system. Applicants have further realized that the systems of the prior art are limited in the type of tests they can support and cannot handle changes which may affect both code and web pages or a combination of the two; [0100]; [0247] Conflict resolver 248 may also be used to detect conflicts and dependencies in such merged state maps, warn about them, and resolve them … interactively; [0124] It will be appreciated that an implementation of system 100 may choose to implement some system elements separately of the experiment management system —either completely separately or using an external mechanism in addition (and in parallel) to the experiment management; [0390] It will be appreciated that system 100 may provide multi-tenancy support, allowing multiple distinct user classes to use the same experiment management system infrastructure and conduct experiments in parallel. [Note that experiments are conducted in parallel and reporting a result or information to a user indicates generating an indication of that result for display, to communicate the outcome clearly and effectively to the user and interactive conflict resolution involves with displaying the conflicting data to the user]).
Per claim 3:
Mordo teaches The method as described in claim 1, wherein the first experiment evaluates a first said value of the variable and the second experiment evaluates a second said value of the variable (Mordo, see at least [0324] [0064]; [0061]; [0063]; [0455] coordinate multiple different experiments under a single unified framework with the ability to handle both code and web pages. System 100 may simultaneously set up and run different types of experiments such as A/B tests, multivariate tests, marketing campaigns, feature toggles and system updates allowing for the collection and analysis of the results of the experiments and modifications to the running of the experiments as a result; [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics (such as session length, conversion to paid users etc.). A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [Note that a single variable is experimented to take on different values across separate experiments comparing multiple variants of a single variable]).
Per claim 4:
Mordo teaches The method as described in claim 1, wherein performing the second experiment in parallel with performing the first experiment resolves the conflict for the second experiment (Mordo, see at least [0057] detect and resolve conflicts between different experiments that are running concurrently as well as have the ability to update the parameters of the experiments and baselines according to an analysis of the results; [0351] running conflicting experiments simultaneously but separately … the failed experiments are discard and the conflict is resolved. This may save coordination work, and may make parallel development easier; [0390] multi-tenancy support, allowing multiple distinct user classes to use the same experiment management system infrastructure and conduct experiments in parallel; [Note conducting experiments in parallel resolve conflicts for certain experiment]).
Per claim 5:
Mordo teaches The method as described in claim 1, further comprising identifying a particular entity included in the first set of entities and the third set of entities ( [0099] The system has performed a secondary conflict check (as described below) and it did not result in canceling the experiment X for the given user, e.g. due to conflict with another (higher priority) experiments for the same user; [0266] experiment selector 310 may select an experiment set for a particular user based on the entered population selection criteria; [0272] Experiment analyzer 251 may later correlate these multiple ID to create merged information reports (e.g. merged BI records under the anonymous used ID and the regular user ID of the same user; [0277] test if the user is a part of a user group for which a user group state-map is specified; [0287] Variant modifier 380 may perform variant selection for all experiments open for the particular user which do not have an existing variant selection (as described in more detail herein below); [Note that a particular user can be identified and selected based on the entered population selection criteria and user group state-map]).
Per claim 6:
Mordo teaches: The method as described in claim 5, wherein the particular entity is associated with a first identification for the first channel and a second identification for the second channel (Mordo, see at least [0270] multiple ID's (of different types) for the same user ... a specialized anonymous user ID for repeating anonymous users (e.g. tracking them through an inserted cookie). The regular user ID would only be assigned later when the user actually registers or logs into the system; [0272] Experiment analyzer 251 may later correlate these multiple ID to create merged information reports (e.g. merged BI records under the anonymous used ID and the regular user ID of the same user); Note that a user is associated with user IDs for different channels which are correlated (stitched) later for identify resolution)).
Per claim 7:
Mordo teaches The method as described in claim 6, wherein the first identification and the second identification are associated with the particular entity using identity stitching (Mordo, see at least [0272] Experiment analyzer 251 may later correlate these multiple ID to create merged information reports (e.g. merged BI records under the anonymous used ID and the regular user ID of the same user); Note that correlating multiple IDs for the same user creates a single, merged profile that is precisely what identity stitching (identity resolution) is).
Per claim 8:
Mordo teaches The method as described in claim 5, wherein the third set of entities and the fourth set of entities are formed by segmenting the first set of entities into the third set of entities and the fourth set of entities (Mordo, see at least [0309] Another type of path expansion may involve registered users only. Population expander 247 may split these users “new users” (registered after the experiment begun) and non-new users. Population expander 247 may allow expansion from new users to all registered users; [0310] Population expander 247 may split the users into “1st time anonymous users” and “repeating anonymous users” (detected according to a cookie installed on their machine). Population expander 247 may allow expansion from “1st time anonymous users” to “all anonymous users”; [0350] Active conflict resolver 370 may also divide the relevant user population between the conflicting experiments, e.g. if experiment E1 and E2 should each be tested on 10% of a relevant user population, each would be allocated a separate 10% “block”. Such separation is performed at the running stage; [0381]; [0414] use user base segmentation information; [Note that users can be segmented into different sets of entities]).
Per claim 10:
Mordo teaches The method as described in claim 9, wherein the conflict for the second experiment is resolved by assigning the second experiment to the first namespace (Mordo, see at least [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based … the experiment scope (i.e. affected system area)—assuming some system areas are more important than others, the running time of the experiment and how close the experiment is to termination (based on achieving a statistically significant result); [0281]; [0285] It will be appreciated that active conflict resolver 370 may detect any conflicts in the calculated experiment assignment (not resolved before) and break each conflict by either using experiment separation so to run each of the experiment on different population (e.g. based on ID) as detailed below or by dropping one of the experiments (with the lower experiment priority; [Note that conflicts detected for experiments are resolved by experiments assignment]).
11. A system comprising: a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising: receiving a request to perform a first experiment via a first channel to evaluate first values of a variable, the first experiment assigned to a first namespace defining a mutually exclusive set of entities (Mordo, see at least [0175] experiments E1 and E2 both modify the same website building system objects components;[0264] by limiting experiments to these whose scope is that of the specific sub-system used (e.g. handle only experiments related to the system editor); [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based (for example) on a combination of any of the following: a priority assigned by the developer, a priority assigned by the experiment manager, the experiment scope (i.e. affected system area)—assuming some system areas are more important than others; [0188] the variant selection for both experiments may be done according to the leading experiment's ID; [0181], [0122]; [0258]; [0181] Experiment A changes a given button to the color red is mutually exclusive (<ME> below) with experiment B which changes the background of this button to the color red. Such exclusion would be required even if the button drawing and background drawing are handled by separate and independent parts of the system; TABLE-US-00001 Category Code Relationship Description Inclusion/ <MI> Must include If [A <MI> B], than A may only be applied exclusion if B is applied as well - either before or after A; [Note that limiting experiments to a specific scope is a form of namespace, where logical grouping (scope) for experiments prevents conflicts and manages access, creating isolated environments. Limiting experiments with unique IDs to the specific context, like the system editor creates a distinct scope for the experiments isolating experiment ID within a defined boundary where the editor experiments live within their space separate from the core system’s entities. An experiment is limited to its own experiment scope (namespace). The system flags the experiment A and B as mutually exclusive because letting both run together would product conflicting result and the active conflict resolver 370 resolves <ME> conflicts using experiment separation by splitting or dropping one of the two experiments].
receiving, via a network, a request to perform a second experiment via a second channel to evaluate second values of the variable, the second experiment assigned to a second namespace defining a mutually exclusive set of entities (Mordo, see at least 0258]; [0181] Experiment A changes a given button to the color red is mutually exclusive (<ME> below) with experiment B which changes the background of this button to the color red. Such exclusion would be required even if the button drawing and background drawing are handled by separate and independent parts of the system; TABLE-US-00001 Category Code Relationship Description Inclusion/ <MI> Must include If [A <MI> B], than A may only be applied exclusion if B is applied as well - either before or after A; [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics … A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [0186]; [0221]; [0257]; [0277]; [0310]; [0381]; [0251]; [0267]; [0442]; [0056] support developer and user supplied requests for how an experiment should be run and the variants used. …a user to request to participate in a particular experiment; [0265] determine which experiments to configure for a given user based on user group state maps; [0277]; [0133] Experiment running (430) (performed by experiment manager 244)—running the system with active experiments (which are applied to entering users) while it collects the results. …when a given experiment reached a pre-specified number Y of users (which received variant B instead of the regular B); [Note that users are segmented into different sets of entities and experiments can be performed via different channels using particular groups of users]).
detecting a conflict for the second experiment based on the first namespace and the second namespace as evaluating a value of a same variable; (Mordo, see at least [0032] Moreover, in accordance with a preferred embodiment of the present invention, the experiment is at least one of: an AB test experiment; [0036] the detecting and resolving resolves conflicts between the experiment and at least one other experiment using experiment separation; [0057]; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122]; [0198]; [0199]; [0285] active conflict resolver 370 may detect any conflicts in the calculated experiment assignment (not resolved before) and break each conflict by either using experiment separation so to run each of the experiment on different population (e.g. based on ID) as detailed below or by dropping one of the experiments (with the lower experiment priority); [0174] Conflict resolver 248 may also check for ambiguity and/or collisions. …. When multiple experiments modify different methods of the same module, system 100 may be configured to classify this as collision or not (i.e. configured for different collision checking granularity); [0175] experiments E1 and E2 both modify the same website building system objects components (and in particular the same attribute) and therefore may also create a problem case. This may be extended to contained objects (e.g. experiment E1 modifies the color of the component X and experiment E2 hides the container containing X); [0176]; [0181]; [0217]; [0236]; [0410]; [0091] A/B tests may support user experience consistency where the same variant selection is used whenever the user re-enters the system; Note that a variable refers to an element on a website or application that can have multiple states such as font size, color, button action etc. and the collision/ambiguity happens when wo experiments try to modify or evaluate the exact same variable for the same user because both experiments are trying to assign a value to the same variable, such as in the A/B test).
resolving the conflict for the second experiment by assigning the second experiment to the first namespace; (Mordo, see at least [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122] experiment managers may work with the UI server which would use an API provided by an experiment handling server. Thus experiment server 240 may also serve as a communication interface allowing developers users etc. to input specifications, state maps, modifications etc., but also to receive warnings, BI reports etc. in order to promote decision making; [0054] Applicants have realized that an experiment management system is desirable for managing multiple experiments inside a component based interactive system such as a website building system. Applicants have further realized that the systems of the prior art are limited in the type of tests they can support and cannot handle changes which may affect both code and web pages or a combination of the two; [0100]; [0247] Conflict resolver 248 may also be used to detect conflicts and dependencies in such merged state maps, warn about them, and resolve them … interactively; [0324] [0064]; [0061]; [0063]; [0455] coordinate multiple different experiments under a single unified framework with the ability to handle both code and web pages. System 100 may simultaneously set up and run different types of experiments such as A/B tests, multivariate tests, marketing campaigns, feature toggles and system updates allowing for the collection and analysis of the results of the experiments and modifications to the running of the experiments as a result; [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics (such as session length, conversion to paid users etc.). A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [Note that a single variable is experimented to take on different values across separate experiments comparing multiple variants of a single variable]).
and performing the second experiment via the second channel to evaluate the second values of the variable in parallel with the first experiment (Mordo, see at least [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122] experiment managers may work with the UI server which would use an API provided by an experiment handling server. Thus experiment server 240 may also serve as a communication interface allowing developers users etc. to input specifications, state maps, modifications etc., but also to receive warnings, BI reports etc. in order to promote decision making; [0054] Applicants have realized that an experiment management system is desirable for managing multiple experiments inside a component based interactive system such as a website building system. Applicants have further realized that the systems of the prior art are limited in the type of tests they can support and cannot handle changes which may affect both code and web pages or a combination of the two; [0100]; [0247] Conflict resolver 248 may also be used to detect conflicts and dependencies in such merged state maps, warn about them, and resolve them … interactively; [0124] It will be appreciated that an implementation of system 100 may choose to implement some system elements separately of the experiment management system —either completely separately or using an external mechanism in addition (and in parallel) to the experiment management; [0390] It will be appreciated that system 100 may provide multi-tenancy support, allowing multiple distinct user classes to use the same experiment management system infrastructure and conduct experiments in parallel. [Note that experiments are conducted in parallel and reporting a result or information to a user indicates generating an indication of that result for display, to communicate the outcome clearly and effectively to the user and interactive conflict resolution involves with displaying the conflicting data to the user]).
14. The system as described in claim 11, wherein the operations further comprise: receiving a request to perform a third experiment using a set of entities used to perform the first experiment; and performing the third experiment using the set of entities used to perform the first experiment (Mordo, see at least [0248] Each user invoking system 100 may receive a different version of the system based on explicit invocation of a specific state map; [0251] Users may receive the B-variant of a component (different from the deployed A-variant) due to AB-testing of an experiment involving the specific component;[0267] a user may enter system 100 and may request a copy of the system client code 50 to run on his user machine 10 from system server set 200; [0056] support developer and user supplied requests for how an experiment should be run and the variants used. …a user to request to participate in a particular experiment; [0265] determine which experiments to configure for a given user based on user group state maps; [0277] test if the user is a part of a user group for which a user group state-map is specified. If so experiment coordinator 330 may activate the state-map and set of experiments accordingly; [0381] If a specific experiment E has proved to be a success, it would be added (eventually) to all users, including users in the “A group” who didn't see it before; [0133] (which received variant B instead of the regular B); [0099]; [0266] experiment selector 310 may select an experiment set for a particular user based on the entered population selection criteria; [0272] Experiment analyzer 251 may later correlate these multiple ID to create merged information reports (e.g. merged BI records under the anonymous used ID and the regular user ID of the same user; [0277] test if the user is a part of a user group for which a user group state-map is specified; [0287] Variant modifier 380 may perform variant selection for all experiments open for the particular user which do not have an existing variant selection (as described in more detail herein below); [Note that a particular user can be identified and selected based on the entered population selection criteria and user group state-map]).
15. The system as described in claim 14, wherein the third experiment is assigned to the second namespace (Mordo, see at least [0248] Each user invoking system 100 may receive a different version of the system based on explicit invocation of a specific state map; [0251] Users may receive the B-variant of a component (different from the deployed A-variant) due to AB-testing of an experiment involving the specific component;[0267] a user may enter system 100 and may request a copy of the system client code 50 to run on his user machine 10 from system server set 200; [0442] System 100 may report the final outcome to the person requesting the modification; [0056] support developer and user supplied requests for how an experiment should be run and the variants used. …a user to request to participate in a particular experiment; [0265] determine which experiments to configure for a given user based on user group state maps; [0277] Experiment manager 244 may accept the request and experiment configurer 243 may test if the user is a part of a user group for which a user group state-map is specified. If so experiment coordinator 330 may activate the state-map and set of experiments accordingly; [0381] If a specific experiment E has proved to be a success, it would be added (eventually) to all users, including users in the “A group” who didn't see it before; [0133] Experiment running (430) (performed by experiment manager 244)—running the system with active experiments (which are applied to entering users) while it collects the results. …when a given experiment reached a pre-specified number Y of users (which received variant B instead of the regular B)). [Note that experiments can be assigned to different namespaces(scopes)]).
Per claims 16, and 18-22, they are the medium versions of claims 1-8, respectively, and are rejected for the same reasons set forth in connection with the rejection of claims 1-8 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 9, 12, 13 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Moordo in view of Padala et al. (US20220276953, hereafter padala).
Per claim 9:
Mordo teaches generating a report summarizing results of the first experiment by associating a particular entity from the group of entities with a first identification for the first channel and a second identification for the second channel (Mordo, see at least [0280] conflict resolver 248 which may resolve dependency circles and conflicts at the experiment integration or setup stages through interaction with the developer/integrator, active conflict resolver 370 may resolve all remaining conflicts and issues automatically. Active conflict resolver 370 may resolve based on experiment priority calculated … resolve based number of users affected by this experiment; [0281]; [0285]; [0290] Experiment coordinator 300 may report to experiment analyzer 251 all details of the experiments and variant assignments, as well as details of all circles/conflicts detected and their resolution and experiment implementer 320 may then send this version to client 10 for execution).
Mordo does not explicitly teach the report includes a summary of the results. However
Padala teaches providing a summary (Padala, see at least [0033] When the defined termination condition is satisfied, the test code executing in container instances 138 may terminate testing and generate a report summarizing the results of the tests executed against compiled application source code in application namespace 132). It would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to have combined Padala’s summary with Mordo’s testing system to modify Mordo’s system to combine the summary function as taught by Padala, with a reasonable expectation of success, since they are analogous art because they are from the same field of endeavor related to testing. Combining Padala’s functionality with that of Mordo results in a system that provides a summary of the experiments. The modification would be obvious because one having ordinary skill in the art would be motivated to make this combination to present a user a report summarizing the results of the experiments for convenience (Padala, see at least [0033] When the defined termination condition is satisfied, the test code executing in container instances 138 may terminate testing and generate a report summarizing the results of the tests executed against compiled application source code in application namespace 132).
Per claims 12 and 13, they are the system versions of claims 6 and 9, respectively, and are rejected for the same reasons set forth in connection with the rejection of claims 6 and 9 above.
Per claim 23, it is the medium system version of claim 9, and is rejected for the same reasons set forth in connection with the rejection of claim 9 above.
Examiner’s Note
The Examiner has pointed out particular references contained in the prior art of record within the body of this action for the convenience of the Applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply. Applicant, in preparing the response, should consider fully the entire reference as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Response to Arguments
Applicant's arguments filed 4/14/2026 have been fully considered but they are not persuasive.
The applicant states that Mordo’s experiment scope is fundamentally different from the claimed namespace. Mordo’s experiment scope is merely a system area designation used to identify which part of the software system and experiment affects. In contrast, the claimed namespace "defines a mutually exclusive set of entities." Mordo's experiment scope does not define or ensure mutually exclusive sets of entities. Rather, Mordo's scope is concerned with which software components or subsystems are affected by an experiment, not with ensuring that user populations (entities) are mutually exclusive … This demonstrates that Mordo's experiment scope does not inherently define mutually exclusive entity sets … Mordo does not teach or suggest "a first namespace defining a mutually exclusive set of entities and a second namespace defining a mutually exclusive set of entities" as recited by claim 1 as amended.
In response, Mordo relies heavily on namespaces to draw the boundaries (isolation) of an experiment’s scope, that is, Mordo’s system uses a namespace architecture to manage and isolate an experiment’s scope, instead of all experiments throwing variables into one space, the scope segregation system assigns experiments to distinct namespaces. The scope is the concept which defines the boundary of what an experiment is allowed to handle/view. The namespace is the background concept for the actual folder/label used to enforce that boundary in the code. The experiment’s scope is restricted to a namespace structure to allow parallel experiments in Mordo, the scope defines where and when that namespace is allowed to be active such as which pages which user segments etc. (The users are segmented into different sets of entities and experiments can be performed via different channels using particular groups of users). The isolated scope of an experiment indicates the namespace (Mordo, see at least [0175] experiments E1 and E2 both modify the same website building system objects components;[0264] by limiting experiments to these whose scope is that of the specific sub-system used (e.g. handle only experiments related to the system editor); [0280] Active conflict resolver 370 may resolve based on experiment priority calculated based (for example) on a combination of any of the following: a priority assigned by the developer, a priority assigned by the experiment manager, the experiment scope (i.e. affected system area)—assuming some system areas are more important than others). When Experiment A is isolated into its own logical grouping, it acts as namespace A. Inside the namespace A, the users or entities are split up into mutually exclusive groups. The same thing happens independently inside namespace B for the experiment B. Because namespace A and B represent distinct logical domains, the experiments run in parallel without collision. (Mordo, see at least [0065] A/B test experiments are used to test the effect of a system modification on specific system metrics … A specific part of the user population is directed to use the modified version of the system (the B-variant) instead of the existing system (the A-variant). The test may include more than 2 options (known as Bx-variants, e.g. the B3-variant), and is then called a Multivariate Test (MVT); [0186]; [0221]; [0257]; [0277]).
The applicant further states that Mordo does not teach (2) Detecting ... a conflict for the second experiment based on the first namespace and the second namespace as evaluating a value of a same variable. Mordo's conflict detection is based on code-level collisions and dependency relationships between experiments, not on whether experiments evaluate the same variable. Specifically, Mordo states that "an experiment conflict may occur (for example) when two experiments modify the same method (or methods), or alternatively ... when two experiments modify the same component." … In contrast, the claimed feature detects a conflict "based on the first namespace and the second namespace as evaluating a value of a same variable." This is a fundamentally different type of conflict detection. Mordo's conflict detection is concerned with code-level interactions (e.g., two experiments modifying the same method or component), not with ensuring that experiments evaluating the same variable are properly grouped to maintain mutually exclusive entity sets. Mordo does not teach or suggest detecting a conflict based on namespace assignments "as evaluating a value of a same variable."
In response, Mordo clearly teaches that the conflict resolver detects upon integration of experiment B into the system that B conflicts with an existing experiment A. A variable (entities or components such as font size, color, button action, page layout, backend setting etc.) and the collision/ambiguity happens when wo experiments try to modify or evaluate the exact same variable for the same user because both experiments are trying to assign a value to the same variable, such as in the A/B test. Mordo teaches such a conflict/collision in experiments in an AB test. When the A/B test runs, it changes a variable’s value and the conflict resolver of Mordo checks for interactions and conflicts between experiments trying to modify or target the exact same component for the same user population (Mordo, see at least [0032] Moreover, in accordance with a preferred embodiment of the present invention, the experiment is at least one of: an AB test experiment; [0036] the detecting and resolving resolves conflicts between the experiment and at least one other experiment using experiment separation; [0057]; [0199] conflict resolver 248 detects upon integration of experiment B into the system that B conflicts (for example) with an existing experiment A, the developer is warned; [0122]; [0198]; [0199]; [0285]; [0174] ).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to INSUN KANG whose telephone number is (571)272-3724. The examiner can normally be reached M-TR 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chat Do can be reached at 571-272-3721. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/INSUN KANG/Primary Examiner, Art Unit 2193
Prosecution Insights