DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Phong et al. (US 2020/0241865 A1) in view of Nickolov et al. (US 2017/0034023 A1).
Regarding claim 1, Phong discloses a system (A container-orchestration system (COS) -184) for dynamically routing users (fig. 3B – users 384A-S) to different versions (first app. Version 112 and second/updated app. version 132) of an application in real-time (during runtime) (Phong, figs. 1 & 3B, [0026] a release orchestrator 184 controls the release process of updates to an application, where version 112 represents a first version of the application. During runtime of the first version (112) of the application, an updated/second version of the application 132 may be routed to one or more users -384A-S),
the system (COS 184) comprising:
at least one network communication interface (one or more network interfaces 324 (wireless and/or wired)) (Phong, fig. 3A [0077] the container-orchestration system (COS) -184) includes hardware 320 comprising a set of one or more processor(s) 322, a set of one or more network interfaces 324 (wireless and/or wired), and non-transitory machine-readable storage media 326 having stored therein software 328 (which includes instructions executable by the set of one or more processor(s) 322));
at least one non-transitory storage device (non-transitory machine-readable storage media 326) (Phong, fig. 3A [0077] the container-orchestration system (COS) -184) includes hardware 320 comprising a set of one or more processor(s) 322, a set of one or more network interfaces 324 (wireless and/or wired), and non-transitory machine-readable storage media 326 having stored therein software 328 (which includes instructions executable by the set of one or more processor(s) 322)); and
at least one processing device (processor(s) 322) coupled to the at least one non-transitory storage device (326) and the at least one network communication interface (network interface 324) (Phong, fig. 3A [0077] the container-orchestration system (COS) -184) includes hardware 320 comprising a set of one or more processor(s) 322, a set of one or more network interfaces 324 (wireless and/or wired), and non-transitory machine-readable storage media 326 having stored therein software 328 (which includes instructions executable by the set of one or more processor(s) 322)),
wherein the at least one processing device (container-orchestration system (COS) -184) is configured to (Phong, [0042] the container-orchestration system (COS) -184 is configured to):
determine at least one unintended workflow (failure) associated with a current version (new/second version/132) of an application that is deployed into production environment in a current release cycle (Phong, [0042] the container-orchestration system (COS) -184 includes an engine 170 used to check the performance of a second version of an application. Based on the responses to the test traffic, the engine 170 determines whether one or more of the tests failed. The engine 170 determines a test has failed when the expected HTTPS response is not received (e.g., no response is received, the type and/or content of the response is incorrect) from the second app version 132. If one or more of the tests failed, the engine 170 retrieves the failure rules 177 for the failed tests and determines an appropriate action),
in response to determining the at least one unintended workflow, continuously
monitor and determine initiation of one or more new sessions by a plurality of users (Phong, [0042:0068] based on the responses to the test traffic, the engine 170 determines whether one or more of the tests failed. The engine 170 determines a test has failed when the expected HTTPS response is not received (e.g., no response is received, the type and/or content of the response is incorrect) from the second app version 132. If one or more of the tests failed, the engine 170 retrieves the failure rules 177 for the failed tests and determines an appropriate action. After a failure is determined in the new version of the application, the system continues to monitor the application by matching it to failure rules set for testing the application);
Phong generally discloses automatically routing user to a previous version of the application (Phong [0065-0066] In block 248, the release orchestrator determines a failure in the second version of the application and takes appropriate action (e.g., terminating the release of the update and any notifying subscribing services of the failure. After terminating the release of the update/second version of the application, traffic is routed to the previous/first version of the application).
Phong did not explicitly disclose calculate probability of initiation of the at least one unintended workflow in the one or more new sessions initiated by each of the plurality of users; determine a criticality score for the one or more new sessions initiated by each of the plurality of users based on the probability of initiation of the at least one unintended workflow; and automatically route each of the plurality of users to a previous version of the application associated with a previous release cycle or the current version of the application associated with the current release cycle based on the criticality score for the one or more new sessions initiated by each of the plurality of users.
Nickolov discloses calculate probability of initiation of the at least one unintended workflow in the one or more new sessions initiated by each of the plurality of users (Nickolov [0027; 0222-0224;0366] Parse Red Hat-provided vulnerability database to identify packages and specific versions/releases that are vulnerable vs. those that have patches for these vulnerabilities. Parse glibc test coverage and results status page to determine whether installed versions may expect failures to known bugs/regression problems. Compare installed packages/versions and configurations against the compiled data from external source and report which issues affect a particular system (or group of systems. DataGrid Reliability Index Score (DGRI Score), DGRI Score is programmatically calculated from a large, and growing, set of historical data crowdsourced from many servers. The determination of a DGRI Score may also be affected by other data sources such as publicly available package vulnerability data, package issue-tracking databases, and test results published for particular package versions);
determine a criticality score for the one or more new sessions initiated by each of the plurality of users based on the probability of initiation of the at least one unintended workflow (Nickolov [0366] a collection of two configurations (two versions of an application) may be assigned separate DGRI Scores representing the probability of successfully changing from the first configuration to the second, and from the second configuration to the first, where these changes involve the upgrade, downgrade, installation or removal of software packages. A DGRI score can also represent the probability of success (or another metric characteristic of) a change from one configuration to another); and
automatically route each of the plurality of users to a previous version (downgrade) of the application associated with a previous release cycle or the current version of the application associated with the current release cycle based on the criticality score for the one or more new sessions initiated by each of the plurality of users (Nickolov [0366] a collection of two configurations (two versions of an application) may be assigned separate DGRI Scores representing the probability of successfully changing from the first configuration to the second, and from the second configuration to the first, where these changes involve the upgrade by transitioning to an upgraded version of an application and downgrade by downgrading to a previous version of the application. The application may also be installed or removed. A DGRI score can also represent the probability of success (or another metric characteristic of) a change from one configuration to another).
One of ordinary skill in the art would have been motivated to combine Phong and Nickolov because these teachings are from the same field of endeavor with respect to disclosing techniques for implementing managing versions of an application.
Therefore, before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to incorporate the strategies by Nickolov into the invention of Phong. The motivation would have been to evaluate server system reliability, vulnerability and component compatibility using crowdsourced server and vulnerability data; generating automated recommendations for improving server system metrics; and automatically and conditionally updating or upgrading system, Nickolov, [Abstract].
Regarding claim 2, Phong modified by Nickolov discloses the system of claim 1, wherein determining the at least one unintended workflow comprises:
generating one or more action intent workflows of one or more previous sessions initiated
by at least one user for accessing one or more features of the application (Nickolov [0395] discloses a configuration version handler retrieves a configuration record from the database (or creates a new one as required, including calculating its DGRI Score) and attaches this configuration to the system record (adding the previously known configuration to the configuration history list of the system record together with timestamps indicating the start and end of the use of this configuration). Next the configuration handler updates the previous and new configuration records to decrement the count of systems using the previous configuration and increment this count for the new configuration. If the new configuration is present in the configuration history of this system, then this is a configuration rollback, and the handler generates an internal rollback signal event),
wherein the one or more action intent workflows are associated with usage of the one or more features of the application that were deployed into the production environment in the previous release cycle (Nickolov [0222-0225] Parse Red Hat-provided vulnerability database to identify packages and specific versions/releases that are vulnerable vs. those that have patches for these vulnerabilities. The patches are used to correct bugs/aggression problems in a given version of a configuration. Parse glibc test coverage and results status page may be used to determine whether installed versions may expect failures to known bugs/regression problems. Compare installed packages/versions and configurations against the compiled data from external source and report which issues affect a particular system (or group of systems) [0225] Recommend which versions/configuration can be used to improve performance and security, based on data from external sources),
determining first set of expected workflows associated with the one or more features (vulnerability and configuration quality/reliabilty) that were deployed into the production environment in the previous release cycle (Nickolov [0215] the system extract vulnerability and configuration quality (reliability) information from external sources (as opposed to telemetry collected from the subscriber systems 410; machine parse data (e.g., API, screen-scraping, etc.); compile data from multiple sources to achieve more complete/reliable information. Analyze configurations collected from the Tracked Servers and provide information/score based on the information compiled from external sources. Example Input data: external sources, telemetry data/configurations. Example Output data: list of vulnerabilities, affected and available packages, vulnerability, reliability and other scores. Example Triggering events: used to evaluate systems against data from external sources to complement data from internal modeling and analytics engines),
identifying one or more new requirements associated with the current release cycle for
development of program code of the application, wherein the one or more new requirements are
associated with one or more new features or modifications of the one or more features (Nickolov [0395] discloses a configuration version handler retrieves a configuration record from the database (or creates a new one as required, including calculating its DGRI Score) and attaches this configuration to the system record (adding the previously known configuration to the configuration history list of the system record together with timestamps indicating the start and end of the use of this configuration). Next the configuration handler updates the previous and new configuration records to decrement the count of systems using the previous configuration and increment this count for the new configuration. If the new configuration is present in the configuration history of this system, then this is a configuration rollback, and the handler generates an internal rollback signal event);
determining a second set of expected workflows for the one or more new requirements
associated with the current release cycle (Nickolov [0395] discloses a configuration version handler retrieves a configuration record from the database (or creates a new one as required, including calculating its DGRI Score) and attaches this configuration to the system record (adding the previously known configuration to the configuration history list of the system record together with timestamps indicating the start and end of the use of this configuration). Next the configuration handler updates the previous and new configuration records to decrement the count of systems using the previous configuration and increment this count for the new configuration. If the new configuration is present in the configuration history of this system, then this is a configuration rollback, and the handler generates an internal rollback signal event);
determining delta changes (increment) between the first set of expected workflows of the previous release cycle and the second set of expected workflows of the current release cycle (Nickolov [0395] discloses a configuration version handler retrieves a configuration record from the database (or creates a new one as required, including calculating its DGRI Score) and attaches this configuration to the system record (adding the previously known configuration to the configuration history list of the system record together with timestamps indicating the start and end of the use of this configuration). Next the configuration handler updates the previous and new configuration records to decrement the count of systems using the previous configuration and increment this count for the new configuration. If the new configuration is present in the configuration history of this system, then this is a configuration rollback, and the handler generates an internal rollback signal event);
monitoring and generating one or more new action intent workflows associated with one
or more current sessions initiated by the at least one user for accessing the application after
deployment of the program code associated with the current release cycle into the production
environment (Nickolov [0587] a Scoring Model Update Procedure may continuously and/or periodically monitor for detection of triggering event(s) for initiating one or more Scoring Model Update(s). Examples of different types of events which may trigger Scoring Model Updates may include, but are not limited to, multiple failure events from configurations with the same configuration element). For example, sufficient number of failure signals (or success signals) received that can be correlated to a particular configuration or configuration element (e.g., a package, a configuration parameter, etc.). Failure signals may indicate systems failing to perform properly. Success signals may indicate systems continuing to work properly. The release of a new version of an OS and/or package version, containing new features, bug fixes and/or fixes for vulnerabilities, incompatibilities, etc. Increase or decrease in reported usage of a new or updated configuration element(s) And/or other types of event(s)/condition(s) which satisfy defined threshold criteria for triggering Scoring Model Update(s)); and
determining the at least one unintended (failure) workflow based on the delta changes (incremental change), the second set of expected workflows, and the one or more new action intent workflows (Nickolov [0587] a Scoring Model Update Procedure may continuously and/or periodically monitor for detection of triggering event(s) for initiating one or more Scoring Model Update(s). Examples of different types of events which may trigger Scoring Model Updates may include, but are not limited to, multiple failure events from configurations with the same configuration element). For example, sufficient number of failure signals (or success signals) received that can be correlated to a particular configuration or configuration element (e.g., a package, a configuration parameter, etc.). Failure signals may indicate systems failing to perform properly. Success signals may indicate systems continuing to work properly. The release of a new version of an OS and/or package version, containing new features, bug fixes and/or fixes for vulnerabilities, incompatibilities, etc. Increase or decrease in reported usage of a new or updated configuration element(s) And/or other types of event(s)/condition(s) which satisfy defined threshold criteria for triggering Scoring Model Update(s)).
The motivation to combine is similar to that of claim 1.
Regarding claim 3, Phong modified by Nickolov disclose the system of claim 2, wherein the at least one processing device is configured to generate the one or more action intent workflows based on:
extracting one or more action logs associated with one or more actions of the at least one user while accessing the one or more features of the application that were deployed in the previous release cycle (Phong [0044] At (8), the release orchestrator 184 performs operations in response to the instructions from the engine 170. When the instructions indicate to transition the sending of the production traffic from the first app version 112 to the second app version 132 (e.g., either no tests failed, or no tests failed with a failure rule blocking the transitioning), the release orchestrator 184 generates updated configuration data for sending to the routing engine 191 (see line 133). The configuration data indicates to the routing engine 191 that it should transition to sending future production traffic 104 from HTTPS clients 107 to the second app version 132 instead of the first app version 112),
identifying one or more corresponding code changes associated with each of the one or
more actions recorded in the action logs (test results log 161) (Phong, [0043] At (7), the engine 170 generates instructions on whether to transition the sending of the production traffic from the first app version 112 to the second app version 132 based on the results of the tests and any applicable failure rules. While in one implementation the engine 170 sends the instructions directly to the release orchestrator 184, in other implementation the instructions are stored in a test results log 161 (e.g., that is a storage or data structure external to engine 170 (see line 153) or that is a part of engine 170) that is accessible to the release orchestrator 184 (e.g., see line 155)); and
generating the one or more action intent workflows based on the one or more action logs
and the one or more corresponding code changes (Phong, [0043] At (7), the engine 170 generates instructions on whether to transition the sending of the production traffic from the first app version 112 to the second app version 132 based on the results of the tests and any applicable failure rules. While in one implementation the engine 170 sends the instructions directly to the release orchestrator 184, in other implementation the instructions are stored in a test results log 161 (e.g., that is a storage or data structure external to engine 170 (see line 153) or that is a part of engine 170) that is accessible to the release orchestrator 184 (e.g., see line 155).
The motivation to combine is similar to that of claim 1.
Regarding claim 4, Phong modified by Nickolov disclose the system of claim 2, wherein the at least one processing device is configured to determine the first set of expected workflows associated with the previous release cycle based on one or more previous requirements associated with the previous release cycle and one or more previous test cases associated with the one or more previous requirements (Phong [0125] discloses the generation and transmission of test traffic that mimics production traffic; and 2) the test traffic to be sent to the second application version while the production traffic continues to be sent to the first application version. In other implementations, this further includes causing: 1) the selection of one or more tests based on an application version identifier of the second app version and version rules prior to generating the test traffic).
The motivation to combine is similar to that of claim 1.
Regarding claim 5, Phong modified by Nickolov disclose the system of claim 4, wherein the at least one processing device is configured to determine the second set of expected workflows based on the one or more new requirements (test features changed/added to the second app version 132) associated with the current release cycle, one or more new test cases associated with the one or more new requirements of the current release cycle, and the one or more previous test cases associated with the one or more previous requirements (Phong [0029] To facilitate the validation process, the engine 170: 1) receives instructions from the release orchestrator 184 to initiate a validation service for the second app version 132; 2) automatically determines one or more tests to apply to the second app version 132 using version information for the second app version 132 and version rules 175; 3) generates and sends test traffic via an HTTPS request to an HTTPS endpoint (e.g., a URL) to the routing engine 191, including an indicator (e.g., in a header of the test traffic) which the routing engine 191 recognizes as indicating that the received traffic is test traffic and should be directed to the second app version 132; and 4) generates instructions on whether to transition the sending of production traffic from the first app version 112 to the second app version 132 based on failure rules and the responses to test traffic. The engine 170 can generate the test traffic to mimic production traffic and to test features changed/added to the second app version 132).
The motivation to combine is similar to that of claim 1.
Regarding claim 6, Phong modified by Nickolov disclose the system of claim 2, wherein the at least one processing device is configured to generate one or more new action intent workflows (test features changed/added to the second app version 132) associated with one or more current sessions initiated by the at least one user for accessing the application after deployment of the program code associated with the current release cycle into the production environment based on extracting one or more current action logs (test results log 161) associated with one or more current actions of the at least one user (administrator) while accessing the application comprising the program code that is deployed in the current release cycle (Phong, [0043;0029] an engine 170 configured by an administrator generates instructions on whether to transition the sending of the production traffic from the first app version 112 to the second app version 132 based on the results of the tests and any applicable failure rules. The instructions are stored in a test results log 161 (e.g., that is a storage or data structure external to engine 170 (see line 153) or that is a part of engine 170) that is accessible to the release orchestrator 184 (e.g., see line 155). To facilitate a validation process, the engine 170: 1) receives instructions from the release orchestrator 184 to initiate a validation service for the second app version 132; 2) automatically determines one or more tests to apply to the second app version 132 using version information for the second app version 132 and version rules 175; 3) generates and sends test traffic via an HTTPS request to an HTTPS endpoint (e.g., a URL) to the routing engine 191, including an indicator (e.g., in a header of the test traffic) which the routing engine 191 recognizes as indicating that the received traffic is test traffic and should be directed to the second app version 132; and 4) generates instructions on whether to transition the sending of production traffic from the first app version 112 to the second app version 132 based on failure rules and the responses to test traffic. The engine 170 can generate the test traffic to mimic production traffic and to test features changed/added to the second app version 132).
The motivation to combine is similar to that of claim 1.
Regarding claim 7, Phong modified by Nickolov disclose the system of claim 1, wherein the at least one processing device is configured to calculate the probability of initiation of the at least one unintended workflow in the one or more new sessions initiated by each of the plurality of users based on: extracting historical action intent workflows associated with each of the plurality of users; generating one or more future action intent workflows for each of the plurality of users based on the historical action intent workflows (Nickolov [0395] discloses a configuration version handler retrieves a configuration record from the database (or creates a new one as required, including calculating its DGRI Score) and attaches this configuration to the system record (adding the previously known configuration to the configuration history list of the system record together with timestamps indicating the start and end of the use of this configuration). Next the configuration handler updates the previous and new configuration records to decrement the count of systems using the previous configuration and increment this count for the new configuration. If the new configuration is present in the configuration history of this system, then this is a configuration rollback, and the handler generates an internal rollback signal event);
calculating probabilities of each intent in the one or more future action intent workflows for each of the plurality of users; and calculating the probability of initiation of the at least one unintended workflow based on the probabilities of the each intent in the one or more future action intent workflows (Nickolov [0825; 0895 -0900] a reliability estimate may be based on the probability of all package upgrades being successful, and may be calculated based on the probability of each package upgrade to be successful (as may be recorded by telemetry data received from various customer servers making these and/or similar version changes. FIG. 31, GUI 3101 may be configured or designed to display various information 3110 related to each source and target package in the change set for the server group such as. A rating (e.g., star rating) for the package representing the package DGRI score. The numeric package DGRI score. The number of package data points (e.g., the number of servers known by the DataGrid System to use, or to have used, this specific package; e.g., as may be used to make the calculation of success probability). The predicted or estimated probability of a successful change from each package from the source version to the target version).
The motivation to combine is similar to that of claim 1.
Regarding claim 8, Phong modified by Nickolov disclose the system of claim 1, wherein the at least one processing device is configured to automatically route each of the plurality of users to the previous version of the application associated with the previous release cycle for the one or more new sessions initiated by each of the plurality of users based on determining that the criticality score is above a defined threshold value (Nickolov [0366;0438 -0441] discloses a target configuration DGRI Score, or the score differential, or the number of data points associated to the target configuration. Such a policy may also be combined with conditionally prompted user input. For example, a policy may define a minimum score, below which the configuration change may not be allowed. A score threshold, above which the user may not be prompted [0441] in between these two, the user is prompted. A collection of two configurations (two versions of an application) may be assigned separate DGRI Scores representing the probability of successfully changing from the first configuration to the second, and from the second configuration to the first, where these changes involve the upgrade by transitioning to an upgraded version of an application and downgrade by downgrading to a previous version of the application. The application may also be installed or removed. A DGRI score can also represent the probability of success (or another metric characteristic of) a change from one configuration to another).
The motivation to combine is similar to that of claim 1.
Regarding claim 9, Phong modified by Nickolov disclose the system of claim 1, wherein the at least one processing device is configured to automatically route each of the plurality of users to the current version of the application associated with the current release cycle for the one or more new sessions initiated by each of the plurality of users based on determining that the criticality score is below a defined threshold value (Nickolov, fig. 61, [1201] discloses DGRI score for different versions of server configuration. A user prompt may be invoked only if the minimum required DGRI score is not met, or if the score change is close to a specified threshold value (e.g., if the score is below a first minimum threshold value, the installation is automatically cancelled; when the score is above a second minimum threshold value, the installation is automatically completed).
The motivation to combine is similar to that of claim 1.
Regarding claim(s) 10-16, the claim(s) is/are rejected with rational similar to that of claim(s) 1-7, respectively.
Regarding claim(s) 17-20, the claim(s) is/are rejected with rational similar to that of claim(s) 1-2 and 7-8, respectively.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. The following publications show the state of the art related to management of versions of an application.
Bonas (US 2020/0259928 A1)
Chen et al. (US 2019/0289057 A1)
Wagner et al. (US 9,715,402 B2)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DIXON F DABIPI whose telephone number is (571)270-3673. The examiner can normally be reached on Monday - Friday from 9:00 am – 5:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher L Parry, can be reached at telephone number 571-272-8328. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center to authorized users only. Should you have questions about access to the USPTO patent electronic filing system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via a variety of formats. See MPEP § 713.01. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/InterviewPractice.
/D.F.D/ Examiner, Art Unit 2451
/GLENFORD J MADAMBA/Primary Examiner, Art Unit 2451
Prosecution Insights