DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Applicant’s claim for the benefit of prior-filed applications 17/510,788, now issued as US 11,893,261, and 18/421,649, now issued as US 12,436,707 under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Acknowledgment is made of applicant's claim for foreign priority based on applications IN202141020578 and IN202141020579, both filed in India on May 5, 2021.
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date under 35 U.S.C. 112(a) as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of the prior-filed applications fail to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for one or more claims of this application.
Upon a review of all applications to which the instant application claims the benefit of, none of the disclosures specifically disclose bi-directional synchronous replication between consistency groups, as recited in the independent claims. While the parent applications do disclose configuring a direction of the replication, this is not sufficient to establish possession of bi-directional synchronous replication as required.
Consequently, the claims do not receive the benefit of the priority claims, and all claims are examined with an effective filing date of January 22, 2025.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Objections
Claims 2-8, 13, and 18 are objected to because of the following informalities:
Claims 2-8 recite “further comprises” but should either recite “further comprising” or “where the method further comprises” for grammatical correctness,
Claim 6, 13, and 18 recite “when the CSN of secondary storage site” but should recite “when the CSN of the secondary storage site” for consistency,
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5, 6, 12, 13, and 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 5 and 12 recite, using claim 5 for example language, “updating the CSN to be equal to the received CSN when the CSN of the primary storage site is less than the received CSN from the secondary storage site”, but “the CSN” lacks proper antecedent basis, as it is unclear by itself whether this refers to the CSN of the primary storage site or the CSN of the secondary storage site. From the context of the claim language and for the purpose of examination, it is assumed this recites “updating the CSN of the primary storage site”.
Claims 6, 13, and 18 recite, using claim 6 for example language, “updating the CSN of the secondary storage site to be equal to the CSN of the primary storage site when the CSN of the secondary storage site is lower than the received CSN”, but “the received CSN” lacks proper antecedent basis, as in the context of claims 1 and 6, there is no CSN that is recited to be received. From the context of the claim language and for the purpose of examination, it is assumed this recites “the CSN of the primary storage site”.
Claim Interpretation
MPEP 2111.04(II) provides that “The broadest reasonable interpretation of a method (or process) claim having contingent limitations requires only those steps that must be performed and does not include steps that are not required to be performed because the condition(s) precedent are not met. For example, assume a method claim requires step A if a first condition happens and step B if a second condition happens. If the claimed invention may be practiced without either the first or second condition happening, then neither step A or B is required by the broadest reasonable interpretation of the claim. If the claimed invention requires the first condition to occur, then the broadest reasonable interpretation of the claim requires step A. If the claimed invention requires both the first and second conditions to occur, then the broadest reasonable interpretation of the claim requires both steps A and B. ”
Claim 5 recites “updating the CSN of the primary storage site to be equal to the received CSN when the CSN of the primary storage site is less than the received CSN from the secondary storage site”. The condition “the CSN of the primary storage site is less than the received CSN from the secondary storage site” is not required within the broadest reasonable interpretation of the claim, as the prior limitation only recites comparing the CSN’s, with no specific result recited. Consequently, the updating limitation is also not required within the broadest reasonable interpretation of the claim.
Claim 6 recites “updating the CSN of the secondary storage site to be equal to the CSN of the primary storage site when the CSN of secondary storage is lower than the CSN of the primary storage site”. The condition “the CSN of the secondary storage site is lower than the CSN of the primary storage site” is not required within the broadest reasonable interpretation of the claim and consequently, the updating limitation is also not required within the broadest reasonable interpretation of the claim.
Claim 8 recites “sending the missing Op from the primary storage site to the secondary storage site when the Op is missing from the second Op log metafile”. The condition “the Op is missing from the second Op log metafile” is not required within the broadest reasonable interpretation of the claim, as the determining recites “determining whether an Op being considered is missing from one of the first and second Op log metafiles” but does not recite the specific result where the Op is missing in the second Op log. Consequently, the sending limitation is not required within the broadest reasonable interpretation of the claim.
For all three claims, applicant can address the respective interpretation by including an additional limitation reciting the condition as positively required or to remove the condition from the contingent limitation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 7, 8, 9, 14, 15, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Subramanian et al. (US 2024/0311033) in view of Chatterjee et al. (US 7,809,892) and Lubbers et al. (US 2003/0188218).
Regarding claim 1, Subramanian teaches a computer-implemented method performed by one or more processing resources of a distributed storage system, the computer-implemented method comprising:
establishing bi-directional synchronous replication between one or more storage objects of a first consistency group (CG1) of a primary storage site and one or more storage objects of a second consistency group (CG2) of a secondary storage site with each storage site having read/write access (Fig. 5 shows two storage systems, with a site A labeled 2102 and site B labeled 2104, where volume V1 on device R1 2124 reads on CG1 and volume V2 on device R2 2126 reads on CG2; regarding bi-directional synchronous replication, Subramanian discloses “FIG. 5 is an example illustrating an active-active replication configuration of a stretched volume using two-way or bidirectional synchronous replication in at least one embodiment in accordance with the techniques of the present disclosure,” [0018], “The example 2500 illustrates an active-active configuration as can be used in connection with synchronous replication in at least one embodiment. In the active-active configuration with synchronous replication, the host 2110a can have a first active path 2108a to the R1 data storage system and R1 device 2124 configured as LUN A. Additionally, the host 2110a can have a second active path 2504 to the R2 data storage system and the R2 device 2126 configured as the same LUN A,” [0082], Subramanian describes two examples, where if a write is directed along link 2108a to R1, then the R1 system sends a write to the R2 system, see [0084], and if a write is directed to R2, then the write is forwarded to the R1 system, see [0085], teaching bidirectional synchronous replication);
initiating a resynchronization process between the one or more storage objects of the CG1 and the one or more storage objects of the CG2 due to a loss of the bi-directional synchronous replication between the one or more storage objects of the CG1 and the one or more storage objects of the CG2 (Figs. 7 and 8 depict resynchronization processes 300 and 400, depending on if site A goes down or site B goes down, reading on resynchronization due to a loss of the bi-directional synchronous replication between CG1 and CG2); and
performing the resynchronization process between the one or more storage objects of the CG1 and the one or more storage objects of the CG2 based on using inflight tracking replay and reconciliation between a first metafile of the primary storage site and a second metafile of the secondary storage site (in the context of Fig. 7, when site A goes down, site B utilizes in-program map of in-progress writes, reading on the inflight tracking replay, applies this to V2 and compares to initial snapshot B1 and post-replay snapshot B2 in steps 302,306,310, as well as utilizing the modified blocks from site B to apply to snapshot A1 of site A in site 312, where the snapshots reads on the metafiles; in the context of Fig. 8, when site B goes down, snapshot A1 is created for site A that performs writes based on an in-progress map in steps 404 and 408, and then when site B is recovered, snapshots B1 and A2 are created in step 410, where modified blocks between A1 and A2 are compared to identify data to be applied to snapshot B1 in steps 412 and 414).
Subramanian fails to explicitly teach where the bidirectional synchronous replication occurs while maintaining zero recovery point objective (RPO) and Zero recovery time objective (RTO), as well as where the metafiles are specifically op logs. While Subramanian’s snapshots do reflect writes that have been finished at different sites, see [0137] these are based on the changes to the data itself, see [0025] not a log of the operations as recited in the claims.
Chatterjee’s disclosure relates to data replication and as such comprises analogous art in the same field of endeavor of data replication.
As part of this disclosure, when discussing the background of different types of replication, Chatterjee states that “Different solutions have been built and deployed that attempt to provide an appropriate RPO and RTO for the particular data storage scenario. As an example, the costliest form of replication, but which has both an RPO and RTO of zero, is called active-active clustering/mirroring. In this form of replication, both the primary and the secondary servers are active and functioning at the same time; clients connect to both of the servers, and the servers maintain consistency with each other at all times. When a primary server fails, the secondary server seamlessly takes over the entire functionality of the system without necessitating a manual fail-over,” Col. 2, Lines 11-21, where RTO and RPO are defined as the recovery time objective and recovery point objective in Col. 1, Line 63 – Col. 2, Line 10.
An obvious modification can be identified: incorporating Chatterjee’s disclosure that active-active replication achieves zero RPO and zero RTO. Such a modification reads upon maintaining an RPO and RTO of zero.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate Chatterjee’s teaching that active-active replication achieves zero RPO and zero RTO into Subramanian’s replication system, as the active-active replication allows for a client to maintain full connection all the time without loss of connection, achieving a high performance metric in RPO and RTO.
The combination of Subramanian and Chatterjee still fails to teach where the metafile used in resynchronization is specifically an op log.
Lubbers’ disclosure is related to active-active data replication, and as such comprises analogous art in the same field of endeavor of data replication.
As part of this disclosure, Lubbers discloses “In an active-active mode, any virtual disk in a copy set may initiate a write/copy operation, and so the group sequence numbering mechanisms allow each site to maintain its own group sequence. Each site then processes the group sequences from each other site in the DRM group to ensure write ordering. An important use for the group sequence mechanism is for ensuring proper write ordering during retransmissions that occur as a result of dropped frames, link failures, or site (controller failures). In retransmission events, a controller receiving the retransmission process only missing operations from the retransmission and discard any retransmitted operations that it has already received and/or processed. Hence, the source controller for the transmission does not need to know which operations are missing for each controller, but can instead replay a log of write/copy operations in order as defined by the sequence number,” [0063].
An obvious modification can be identified: incorporating a log of write/copy operations to replay for each controller into Subramanian’s replication system. Such a modification reads upon the use of an op log metafile specifically for the resynchronization process.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate Lubbers’ operation log into Subramanian’s replication system, as an operation log can be used to ensure that any missing writes from a failed site are processed in a correct order to leave the site in a correct state, not just computing snapshot differences.
Regarding claim 7, the combination of Subramanian, Chatterjee, and Lubbers teaches the computer-implemented method of claim 1, further comprising:
initiating the inflight tracking and reconciliation of I/O operations based on the first Op log metafile of the primary storage site and the second Op log metafile of the secondary storage site (Subramanian Figs. 7 and 8 identify different snapshots on site A and site B to compare in order to perform reconciliation of I/O operations, see the steps identified in the claim 1 rationale; the claim 1 rationale then incorporates operation logs from Lubbers, as disclosed in the claim 1 rationale).
Regarding claim 8, the combination of Subramanian, Chatterjee, and Lubbers teaches the computer-implemented method of claim 1, further comprising:
replaying the I/O operations from one of the first and second Op log metafiles (Subramanian Fig. 7 and 8 show different embodiments of applying the in-progress map to either a site A or site B snapshot, incorporating Lubbers in the claim 1 rationale to teach utilizing replaying the I/O operations log); and
reconciling the I/O operations in the first Op log metafile with the I/O operations in the second Op log metafile by determining whether an Op being considered is missing from one of the first and second Op log metafiles (Subramanian Figs. 7 and 8 depict a process of identifying differences in data between a site A snapshot and a site B snapshot, see Fig. 7, steps 310,312,314 and Fig. 8, steps 410,412,414,416, with the modification from Lubbers providing for determining if an op is missing, not just if data differences still exist and sending the missing Op from the primary storage site to the secondary storage site when the Op is missing from the second Op log metafile (while not considered necessary to teach due to the contingent nature of the limitation, in the case of Fig. 8 where site B goes down, then ops would be missing from site B’s snapshots/replay logs, so Subramanian Fig. 8 steps 414 and 416 identifies modified blocks to be sent from site A and site B).
Regarding claim 9, Subramanian teaches A non-transitory computer-readable storage medium embodying a set of instructions , which when executed by one or more processing resources of a multi-site distributed storage system cause the one or more processing resources to perform the method of claim 1 (“Various embodiments of the techniques herein can include a computer-implemented method, a system and a non-transitory computer readable medium. … The non-transitory computer readable medium can include code stored thereon that, when executed, performs the method,” [0003]), and can be rejected according to the same rationale.
Claim 14 is rejected according to the rationale of both claims 7 and 8.
Regarding claim 15, Subramanian teaches a multi-site storage system having a primary storage site with a primary storage cluster and a secondary storage site with a secondary storage cluster (“Various embodiments of the techniques herein can include a computer-implemented method, a system and a non-transitory computer readable medium,” [0003], where the multisite storage system is shown in Fig. 5 with a primary storage site and a secondary storage site), comprising:
one or more processing resources (“The system can include one or more processors,” [0003]); and
a non-transitory computer-readable medium coupled to the one or more processing resources, having stored therein instructions which when executed by the one or more processing resources cause the one or more processing resources to perform the method of claim 1 (“The system can include … a memory comprising code that, when executed, performs the method”, [0003]) and rejected according to the same rationale.
Claims 19 and 20 are rejected according to the same rationale of claims 7 and 8 respectively.
Claims 2, 3, 10, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Subramanian in view of Chatterjee and Lubbers and further in view of Vankamamidi et al. (US 11,327,895).
Regarding claim 2, the combination of Subramanian, Chatterjee, and Lubbers teaches the computer-implemented method of claim 1, further comprising:
maintaining counters for Operation Sequence Number on the primary storage site (as cited in the claim 1 rationale, Lubbers [0063] provides for sequence numbers for the operation log, where each site maintains its own group sequence, teaching maintaining a counter in the primary storage site).
The combination fails to teach the method further comprising:
designating the primary storage site as a global sequence number generator to ensure unique sequence numbers for operations received on each of the primary and secondary storage sites; and
maintaining a cumulative response sequence number (CSN) on the primary storage site.
While Subramanian discloses providing one site as a preferred site, see [0028], which allows for different processing of the writes, as well as how a failure in the bidirectional replication affects the preferred/non-preferred sites, see Figs. 7 and 8, and Lubbers does identify designating a site as a group master, see [0082], neither specifically discuss the primary storage site as a global sequence number generator or maintaining counters.
Vankamamidi’s disclosure relates to reconciling sequence numbers for data writes across different nodes, and as such comprises analogous art in the same field of endeavor of data replication.
As part of this disclosure, Vankamamidi provides for two nodes A and B in Fig. 3, where “In one such dual node system, the designated primary node that assigns sequence numbers is also the sole node managing the page descriptors or PDESCs stored in the cache. Thus the primary node is the sole node of the pair that assigns sequence numbers of cached items or operations received by both nodes. The primary node also is the sole node that handles allocating PDESCs and freeing PDESCs on behalf of requests or operations received by both nodes. Thus, in an active-active configuration where both nodes of the pair may receive I/Os and other commands, all requests for new sequence numbers and new PDESC allocations for caching are done only by the primary node even though the received I/Os or other commands may be received by the peer non-primary node,” Col. 14, Lines 13-26.
Vankamamidi also discloses in Fig. 5 where both node A and node B each have a memory that tracks the largest sequence ID assigned by both nodes, see data structures 530 and 540.
An obvious modification can be identified: incorporating Vankamamidi’s disclosure that the primary node is in charge of assigning the sequence numbers for operations and incorporating Vankamamidi’s data structure tracking the largest sequence ID assigned by a node. The primary node assigning sequence number reads on the designation of the primary storage site as the sequence number generator to ensure unique sequence numbers, and the data structure reads on the counter for the CSN on the primary storage site.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate Vankamamidi’s disclosure of having the primary node in charge of assigning sequence numbers, as this ensures that one node takes priority/precedence in the administrative task of assigning and prevents confusion of having the nodes potentially assign conflicting sequence numbers, and it would have further been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate tracking CSN’s within each node, as this provides each node with its own up to date view on what operations have been performed by each node, allowing an easy way to track what operations may be missing with respect to Subramanian’s in-progress map/snapshot resynchronization and Lubbers’ replay log.
Regarding claim 3, the combination of Subramanian, Chatterjee, and Lubbers teaches the computer-implemented method of claim 1, but fails to teach the method further comprising:
maintaining a counter for CSN on the secondary storage site.
Vankamamidi’s disclosure relates to reconciling sequence numbers for data writes across different nodes, and as such comprises analogous art in the same field of endeavor of data replication.
As part of this disclosure, Vankamamidi discloses in Fig. 5 where both node A and node B each have a memory that tracks the largest sequence ID assigned by both nodes, see data structures 530 and 540.
An obvious modification can be identified: incorporating Vankamamidi’s data structure tracking the largest sequence ID assigned by a node. The primary node assigning sequence number reads on the designation of the primary storage site as the sequence number generator to ensure unique sequence numbers, and the data structure reads on the counter for the CSN on the primary storage site.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate Vankamamidi’s tracking CSN’s within each node, as this provides each node with its own up to date view on what operations have been performed by each node, allowing an easy way to track what operations may be missing with respect to Subramanian’s in-progress map/snapshot resynchronization and Lubbers’ replay log.
Claim 10 is rejected according to the same rationale of claims 2 and 3.
Claim 16 is rejected according to the same rationale of claims 2 and 3.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
For clarity of record, all double patenting rejections below only show one the method claims, with the assumption that similar comments/analysis applies to the system/non-transitory computer readable medium claims, as those are obvious variants of each other as embodiments. Further, all limitations are shown in correct order for the instant claims, with limitations of the patented claims moved around as needed.
Claims 1, 7, 8, 9, 14, 15, 19, and 20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 4, 7, 9, 10, and 13 of U.S. Patent No. 11,893,261 in view of Subramanian and Chatterjee.
Claims 2, 3, 10, and 16 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 4, 7, 9, 10, and 13 of U.S. Patent No. 11,893,261 in view of Subramanian, Chatterjee, Lubbers, and Vankamamidi.
Claim 1, instant application
Claim 3, US 11,893,261 (incorporating claim 1)
. A computer-implemented method performed by one or more processing resources of a distributed storage system, the computer-implemented method comprising:
establishing bi-directional synchronous replication between one or more storage objects of a first consistency group (CG1) of a primary storage site and one or more storage objects of a second consistency group (CG2) of a secondary storage site with each storage site having read/write access while maintaining zero recovery point objective (RPO) and Zero recovery time objective (RTO);
initiating a resynchronization process between the one or more storage objects of the CG1 and the one or more storage objects of the CG2 due to a loss of the bi-directional synchronous replication between the one or more storage objects of the CG1 and the one or more storage objects of the CG2; and
performing the resynchronization process between the one or more storage objects of the CG1 and the one or more storage objects of the CG2 based on using inflight tracking replay and reconciliation between a first Op log metafile of the primary storage site and a second Op log metafile of the secondary storage site.
A computer implemented method performed by one or more processing resources of a distributed storage system, the method comprising:
maintaining state information regarding a data replication status for a storage object of a primary storage cluster with the storage object being replicated to a replicated storage object of a secondary storage cluster;
… wherein the resynchronization resumes zero recovery point objective (RPO) protection…
temporarily disallowing input/output (I/O) operations for the storage object when the storage object of the primary storage cluster has a connection loss or failure, which causes an internal state as out of sync for the storage object of the primary storage cluster while maintaining an external state as in sync for external entities in order to provide time for reestablishing synchronous replication within duration of an operation (Op) timeout period;
…
performing a resynchronization between the storage object and the replicated storage object based on the persistent inflight tracking and reconciliation of I/O operations using the first Op log of the primary storage cluster and the second Op log of the secondary storage cluster.
Claim 7, instant application
Claim 1, US 11,893,261
The computer-implemented method of claim 1, further comprises:
initiating the inflight tracking and reconciliation of I/O operations based on the first Op log metafile of the primary storage site and the second Op log metafile of the secondary storage site.
…performing persistent inflight tracking and reconciliation of I/O operations based on a first Op log of the primary storage cluster and a second Op log of the secondary storage cluster…
Claim 8, instant application
Claim 4, US 11,893,261
The computer-implemented method of claim 1, further comprises:
replaying the I/O operations from one of the first and second Op log metafiles; and
reconciling the I/O operations in the first Op log metafile with the I/O operations in the second Op log metafile by determining whether an Op being considered is missing from one of the first and second Op log metafiles and sending the missing Op from the primary storage site to the secondary storage site when the Op is missing from the second Op log metafile.
The computer implemented method of claim 1, further comprising:
replaying the I/O operations from one of the first and second Op logs; and
reconciling the I/O operations in the first Op log with the I/O operations in the second Op log by determining whether an Op being considered is present or absent in both of the first and second Op logs and skipping the Op if the Op is present or absent in both of the first and second Op logs from the primary and secondary storage clusters.
As seen in the table above, while a number of limitations are disclosed in the patented claims, with regards to claims 1, 9, 15, the bi-directional nature of the data replication is not disclosed in the patented claims, nor is the achievement of zero RTO.
These are disclosed in Subramanian Fig. 5, [0082,0084,0085] and Chatterjee Col. 1, Line 63 – Col. 2, Line 21, as relied upon in the rejection under 35 U.S.C. 103.
It would have been obvious to one of ordinary skill in the art to incorporate Subramanian’s disclosure concerning active-active bidirectional data replication, as this ensures that both sites can receive data from the host, even in potential downtimes of one of the sites, without loss of connection from the point of view of the host.
The Chatterjee rational/motivation is identical to what is presented in the rejection under 35 U.S.C. 103.
With regards to claim 8, as the sending limitation has been identified to be contingent, then the patented claim is not required to teach this limitation.
This differs with claims 14 and 20, which do require this limitation. While the majority of the claimed subject matter is taught by the patented claims, as seen above, the patented claims do not teach sending the missing Op from the primary storage site to the secondary storage site when the Op is missing from the second Op log metafile. This is taught by Subramanian as discussed in the claim rejection under 35 U.S.C. 103.
With regards to claims 2, 3, 10, and 16, none of the patented claims disclose this subject matter. However, the combination of Subramanian, Chatterjee, Lubbers, and Vankamamidi teaches these limitations, as discussed in the rejection under 35 U.S.C. 103. For clarity of record, while Lubbers is not considered necessary to reject claim 1 under double patenting grounds, the rationale for incorporating Lubbers in rejecting claim 1 under 35 U.S.C. 103 is incorporated into the rejection for claims 2 and 3.
Claims 1, 7, 8, 9, 14, 15, 19, and 20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 4, 9, 10, and 13 of U.S. Patent No. 12,436,707 in view of Subramanian and Chatterjee.
Claims 2, 3, 10, and 16 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 4, 9, 10, and 13 of U.S. Patent No. 12,436,707 in view of Subramanian, Chatterjee, Lubbers and Vankamamidi.
Claim 1, instant application
Claim 3, US 12,436,707 (incorporating claim 1)
. A computer-implemented method performed by one or more processing resources of a distributed storage system, the computer-implemented method comprising:
establishing bi-directional synchronous replication between one or more storage objects of a first consistency group (CG1) of a primary storage site and one or more storage objects of a second consistency group (CG2) of a secondary storage site with each storage site having read/write access while maintaining zero recovery point objective (RPO) and Zero recovery time objective (RTO);
initiating a resynchronization process between the one or more storage objects of the CG1 and the one or more storage objects of the CG2 due to a loss of the bi-directional synchronous replication between the one or more storage objects of the CG1 and the one or more storage objects of the CG2; and
performing the resynchronization process between the one or more storage objects of the CG1 and the one or more storage objects of the CG2 based on using inflight tracking replay and reconciliation between a first Op log metafile of the primary storage site and a second Op log metafile of the secondary storage site.
A computer implemented method performed by one or more processing resources of a distributed storage system, the method comprising:
… wherein the resynchronization resumes zero recovery point objective (RPO) protection…
initiating a resynchronization process due to a failure or loss of connectivity of a storage object of a primary storage cluster with the storage object becoming out of sync as an internal state while maintaining an external state as in sync for external entities in order to provide time for reestablishing synchronous replication between the storage object and a replicated storage object of a secondary storage cluster within duration of an operation (Op) timeout period; and
performing the resynchronization process between the storage object and the replicated storage object based on using a first Op log of the primary storage cluster and a second Op log of the secondary storage cluster.
Claim 7, instant application
Claim 4, US 12,436,707
The computer-implemented method of claim 1, further comprises:
initiating the inflight tracking and reconciliation of I/O operations based on the first Op log metafile of the primary storage site and the second Op log metafile of the secondary storage site.
The computer implemented method of claim 1, further comprising:
initiating persistent inflight tracking and reconciliation of I/O operations based on the first Op log of the primary storage cluster and the second Op log of the secondary storage cluster…
Claim 8, instant application
Claim 4, US 12,436,707
The computer-implemented method of claim 1, further comprises:
replaying the I/O operations from one of the first and second Op log metafiles; and
reconciling the I/O operations in the first Op log metafile with the I/O operations in the second Op log metafile by determining whether an Op being considered is missing from one of the first and second Op log metafiles and sending the missing Op from the primary storage site to the secondary storage site when the Op is missing from the second Op log metafile.
The computer implemented method of claim 1, further comprising:
…
replaying the I/O operations from one of the first and second Op logs; and
reconciling the I/O operations in the first Op log with the I/O operations in the second Op log by determining whether an Op being considered is present or absent in both of the first and second Op logs and skipping the Op if the Op is present or absent in both of the first and second Op logs from the primary and secondary storage clusters.
As seen in the table above, while a number of limitations are disclosed in the patented claims, with regards to claims 1, 9, 15, the bi-directional nature of the data replication is not disclosed in the patented claims, nor is the achievement of zero RTO.
These are disclosed in Subramanian Fig. 5, [0082,0084,0085] and Chatterjee Col. 1, Line 63 – Col. 2, Line 21, as relied upon in the rejection under 35 U.S.C. 103.
It would have been obvious to one of ordinary skill in the art to incorporate Subramanian’s disclosure concerning active-active bidirectional data replication, as this ensures that both sites can receive data from the host, even in potential downtimes of one of the sites, without loss of connection from the point of view of the host.
The Chatterjee rational/motivation is identical to what is presented in the rejection under 35 U.S.C. 103.
With regards to claim 8, as the sending limitation has been identified to be contingent, then the patented claim is not required to teach this limitation.
This differs with claims 14 and 20, which do require this limitation. While the majority of the claimed subject matter is taught by the patented claims, as seen above, the patented claims do not teach sending the missing Op from the primary storage site to the secondary storage site when the Op is missing from the second Op log metafile. This is taught by Subramanian as discussed in the claim rejection under 35 U.S.C. 103.
With regards to claims 2, 3, 10, and 16, none of the patented claims disclose this subject matter. However, the combination of Subramanian, Chatterjee, Lubbers, and Vankamamidi teaches these limitations, as discussed in the rejection under 35 U.S.C. 103. For clarity of record, while Lubbers is not considered necessary to reject claim 1 under double patenting grounds, the rationale for incorporating Lubbers in rejecting claim 1 under 35 U.S.C. 103 is incorporated into the rejection for claims 2 and 3.
Allowable Subject Matter
Claims 4, 11, 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 5, 6, 12, 13, and 18 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claims 4, 11, and 17 recite, using claim 4 for example language:
upon operations for one or more storage objects of CG1 being received on the primary storage site, assigning the received operations an Op Sequence Number and CSN;
sending the operations having the Op Sequence Number and CSN from the primary storage site to the secondary storage site; and
receiving, with the primary storage site, a response from the secondary storage site having a CSN for the secondary storage site.
Claims 6, 13, and 18 recite, using claim 6 for example language:
receiving operations on the secondary storage site for one or more storage objects of CG2;
sending the received operations with a CSN of the secondary storage site to the primary storage site;
sending a response from the primary storage site having an Op Sequence Number that is generated on the primary storage site and a CSN of the primary storage site to the secondary storage site; and
In both sets of claims, the claims require receiving ops on one site (primary in claims 4/11/17, secondary in claims 6/13/18), sending the operations with a respective CSN to the other site (primary to secondary in claims 4/11/17, secondary to primary in claims 6/13/18), and then receiving a response with a CSN (receiving from the secondary in claims 4/11/17, receiving from the primary in claims 6/13/18). The first two steps are addressed in Vankamamidi, see Col. 21, Line 58 – Col. 23 Line 50 describing an example process, where writes are received at nodes A and B, and how operations and sequence numbers are forwarded to the other nodes to update their sequence number tracking. However, Vankamamidi never provides where the node that the operations/sequence number are forwarded to in turn sends a response back to the originating node. As such, the combination of Subramanian, Chatterjee, and Vankamamidi fails to render the subject matter of claims 4, 6, 11, 13, 17, and 18 obvious, and no other reference is found to render this feature obvious.
The claims also cannot be considered obvious in view of the patent claims at issue in the double patenting rejection, as the double patenting rejection would rely upon a secondary reference, which as discussed has not been found. Consequently, the claims are considered allowable, other than the issues identified under 35 U.S.C. 112(b).
Claims 5 and 12 are indicated to recite allowable subject matter due to their dependence on claims 4 and 11 respectively.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lubbers et al. (US 2003/0188233) specifically discloses tracking sequence numbers for comparison and copying when GSN’s are identified as missing in a replication site,
Rostagni et al. (US 2019/0227888) discloses handling node failures in an active-active configuration and how to reconcile writes between sites,
Leskes (US 2020/0201879) provides an active-passive relationship where one node is selected as leader in charge of providing operations/sequence numbers between nodes,
Chen et al. (US 2020/0233582) discloses transitioning between replication modes, including bidirectional mirroring for replicating consistency groups,
Chen et al. (US 2021/0263649) discloses the use of snapshots and inflight trackers for bidirectional replication,
Chen et al. (US 2022/0197924) discloses active-active replication and the use of journals for data recovery,
Meiri et al. (US 2023/0009529) discloses active-active storage configuration with more disclosure on the replication across storage sites,
Wu et al. (US 2024/0176703) discloses recovering data across consistency groups in active-active replication configuration.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON D HO whose telephone number is (469)295-9093. The examiner can normally be reached Mon-Fri 8:00-4:00 CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Reginald Bragdon can be reached at (571)272-4204. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.D.H./Examiner, Art Unit 2139
/REGINALD G BRAGDON/Supervisory Patent Examiner, Art Unit 2139
Prosecution Insights