DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/10/2025 was filed after the mailing date of the application. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-7, 11-18 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-7, 13-20 of U.S. Patent No. 10,964,118 (Patent ‘118, hereinafter). Although the claims at issue are not identical, they are not patentably distinct from each other because all the features of current claims 1-7, 11-18 are already included in claims 1-7, 13-20 of Patent ‘118. See Tables below.
Table I
Current Application 18/977138
US. Patent No. 10,964,118
1-7, 11-18
1-7, 13-20, respectively
Table II
Current Application 18/977138
US. Patent No. 10,964,118
1. A system of providing augmented reality (AR) with object recognition in a physical environment to advance a narrative, comprising: a processor programmed to:
access narrative data representing the narrative, the narrative data comprising one or more nodes, each node of the one or more nodes being associated with a decision rule that specifies one or more physical objects in the physical environment to be recognized through image recognition to advance past the node in the narrative;
for a first node of the one or more nodes:
generate a user interface associated with the narrative for the first node; access a first decision rule specifying a first physical object to be recognized to advance past the first node in the narrative;
perform, using an image classifier, image recognition on an image of the physical environment;
determine that the first physical object is in the physical environment based on the image recognition;
transition from the first node in the narrative based on the first decision rule and the determination that the first physical object is in the physical environment; and
update the user interface to a second node in the narrative based on the transition from the first node.
1. A system of providing augmented reality (AR) with object recognition in a physical environment to advance a narrative, comprising: a processor programmed to:
access narrative data representing the narrative, the narrative data comprising one or more nodes, each node of the one or more nodes being associated with a decision rule that specifies one or more physical objects in the physical environment to be recognized through image recognition to advance past the node in the narrative;
for a first node of the one or more nodes:
generate a user interface associated with the narrative for the first node, the user interface including a first virtual object and a real image of a first physical object captured by an image capture device in the physical environment, the first virtual object being overlaid onto a view of the physical environment that includes the first physical object and the first virtual object being programmed to react to and interact with, in the user interface, the real image of the first physical object; access a first decision rule specifying the first physical object to be recognized to advance past the first node in the narrative;
perform, using an image classifier, image recognition on an image of the physical environment;
determine that the first physical object is in the physical environment based on the image recognition;
generate, in the user interface, a virtual interaction by the first virtual object with the real image of the first physical object so that the first virtual object is displayed to be overlaid onto the view of the physical environment and the real image of the first physical object captured by the image capture device and interact with the real image of the first physical object captured by the image capture device and imaged in the user interface;
transition from the first node in the narrative based on the first decision rule and the determination that the first physical object is in the physical environment; and
update the user interface to a second node in the narrative based on the transition from the first node, the second node comprising a result of the virtual interaction between the first virtual object and the real image of the first physical object in the user interface.
From the Tables above, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims 1-7, 13-20 of Patent ‘118 to be as currently claims 1-7, 11-18 since all of the features of current claims 1-7, 11-18 are already included in claims 1-7, 13-20 of Patent ‘118.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-9, 11-19 are rejected under 35 U.S.C. 103 as being unpatentable over DeFaria et al. (US. Patent App. Pub. No. 2019/0251751, “DeFaria”, hereinafter) in view of Brent et al. (US. Patent App. Pub. No. 2019/0228269, “Brent”).
As per claim 1, as shown in Fig. 4, DeFaria teaches a system of providing augmented reality (AR) with object recognition in a physical environment to advance a narrative (¶ [60]), comprising: a processor programmed to:
access narrative data representing the narrative, the narrative data comprising one or more nodes (¶ [50], “…a narrative ruleset 412 that defines the pace of different narrative events based on one or more control criteria”. The nodes are interpreted as the events in the group chain shown in Fig. 3, ¶ [45]), each node of the one or more nodes being associated with a decision rule that specifies one or more physical objects in the physical environment to be recognized through image recognition to advance past the node in the narrative (further addressed below referring to Brent. See Fig. 4, ¶ [49-51], decision by the pace controller 406 which content to be displayed based on the ruleset 412 responsive the input. DeFaria does also teach recognizing object in the physical environment as described at ¶ [64]);
for a first node of the one or more nodes:
generate a user interface associated with the narrative for the first node (¶ [40]. See further below referring to Brent).
DeFaria does not expressly teach access a first decision rule specifying a first physical object to be recognized to advance past the first node in the narrative;
perform, using an image classifier, image recognition on an image of the physical environment;
determine that the first physical object is in the physical environment based on the image recognition;
transition from the first node in the narrative based on the first decision rule and the determination that the first physical object is in the physical environment; and
update the user interface to a second node in the narrative based on the transition from the first node.
Brent teaches a similar method of image processing for augmented reality (Abstract, and ¶ [2]), the method further comprises the above features, i.e., access a first decision rule specifying a first physical object to be recognized to advance past the first node in the narrative (¶ [42]);
perform, using an image classifier, image recognition on an image of the physical environment (¶ [192]);
determine that the first physical object is in the physical environment based on the image recognition; transition from the first node in the narrative based on the first decision rule and the determination that the first physical object is in the physical environment (¶ [163], “The interpreter 124 transforms detected physical objects and movements of the detected physical objects into programmatic instruction. The interpreter 124 evaluates the correlations against protocols in the protocol library 108, known objects in the known objects library 112, and declared objects 138, to generate programmatic instruction for controlling and operating an augmented reality device”, i.e., transitioning the first node. See also Fig. 3); and
update the user interface to a second node in the narrative based on the transition from the first node (¶ [53], for example, “…providing visual or other cues for the next operation or object with which the worker should engage”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method as taught by Brent into the method as taught by DeFaria as addressed above, the advantage of which is to enable better human performance (¶ [3]).
As per claim 2, the combined teachings of DeFaria and Brent impliedly teach wherein the first decision rule further specifies a second physical object to be recognized to advance past the first node, wherein the processor is further programmed to: advance past the first node only if the second physical object is also determined to be in the physical environment based on the image recognition (see the determining step in claim 1, referring to Brent ¶ [163], detecting more than one objects). Thus, claim 2 would have been obvious over the combined references for the reason above.
As per claim 3, as addressed in claim 2, the combined DeFaria-Brent does also impliedly teach wherein the first decision rule specifies a plurality of physical objects, including the first physical object, to be recognized to advance past the first node, wherein the processor is further programmed to:
advance to a second node responsive to a determination that a first set of objects among the plurality of objects is in the physical environment (see claim 2); and advance to a third node responsive to a determination that a second set of objects among the plurality of objects is in the physical environment (Brent, ¶ [58], i.e., manipulating the next object in the sequence operation. The detection of the physical objects in the physical environment is described at ¶ [42] of Brent). Thus, claim 3 would have been obvious over the combined references for the reason above.
As per claim 4, the combined DeFaria-Brent substantially teaches wherein the first decision rule further specifies an action to be performed with respect to the first physical object to advance past the first node (Brent, ¶ [53] and ¶ [58], i.e., guiding the user to manipulating the object when the previous step is completed). Thus, claim 4 would have been obvious over the combined references for the reason above.
As per claim 5, the combined DeFaria-Brent does further teach wherein the user interface displays a first virtual object, the action to be performed comprises causing the first physical object to be within a sufficient proximity to a virtual object displayed by the user interface, and wherein the processor is further programmed to: determine whether the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface (Brent, ¶ [58], the object to be manipulated is overlaid with visuals indicating imminent contact or proximity by the user). Thus, claim 5 would have been obvious over the combined references for the reason above.
As per claim 6, the combined DeFaria-Brent sufficiently teaches wherein to determine whether the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface, the processor is further programmed to: determine whether pixels of the first physical object imaged in the user interface is within a threshold number of pixels of the AR object displayed in the user interface (Brent, ¶ [118], “ “Log Action initiated.” Timing starts when user hand enters the volume near a colored overlay (e.g., dot)” in a series of actions prompted). Thus, claim 6 would have been obvious over the combined references for the reason above.
As per claim 7, the combined DeFaria-Brent does also impliedly teach wherein the processor is further programmed to:
determine that the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface (addressed in claims 5 and 6 above);
determine that an interaction has occurred between the first physical object imaged in the user interface and the virtual object in the user interface based on the determination that the that the first physical object imaged in the user interface is within the sufficient proximity to the virtual object in the user interface (as addressed in claim 6); and display a result of the interaction in the user interface (Brent, ¶ [131-133]). Thus, claim 7 would have been obvious over the combined references for the reason above.
As per claim 8, as addressed, the combined DeFaria-Brent does further impliedly teach wherein the first decision rule further specifies a user changed perspective in the physical environment to occur such that the first physical object has a certain size relative to the first physical object (see Brent, ¶ [273], “…an impression can be given of the augmentation being obscured by a real-world object that is in front of its perceived location from the perspective of the user”). Thus, claim 8 would have been obvious over the combined references for the reason above.
As per claim 9, as recited in claim 8, the combined DeFaria-Brent does also teach wherein the first decision rule further specifies an AR-guided interaction to be performed on the first physical object (Brent, ¶ [273], dynamically interacting with a real-world object). Thus, claim 9 would have been obvious over the combined references for the reason above.
Claim 11, which is similar in scope to claim 1 as addressed above, is thus rejected under the same rationale.
Claim 12, which is similar in scope to claim 2 as addressed above, is thus rejected under the same rationale.
Claim 13, which is similar in scope to claim 3 as addressed above, is thus rejected under the same rationale.
Claim 14, which is similar in scope to claim 4 as addressed above, is thus rejected under the same rationale.
Claim 15, which is similar in scope to claim 5 as addressed above, is thus rejected under the same rationale.
Claim 16, which is similar in scope to claim 6 as addressed above, is thus rejected under the same rationale.
Claim 17, which is similar in scope to claim 7 as addressed above, is thus rejected under the same rationale.
Claim 18, which is similar in scope to claim 8 as addressed above, is thus rejected under the same rationale.
Claim 19, which is similar in scope to claim 9 as addressed above, is thus rejected under the same rationale.
Claims 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over DeFaria et al. (US. Patent App. Pub. No. 2019/0251751) in view of Brent et al. (US. Patent App. Pub. No. 2019/0228269) further in view of Ofek et al. (US. Patent App. Pub. No. 2016/0342432, “Ofek”).
As per claim 10, the combined DeFaria-Brent does not explicitly teach wherein the processor is further programmed to: generate a virtual avatar to guide the AR-guided interaction to be performed on the first physical object. However, this is well-known in the art as is disclosed by Ofek, in a similar method of guiding a user to perform a particular task (see Abstract), wherein the method includes generate a virtual avatar to guide the AR-guided interaction to be performed on the first physical object (see ¶ [26-27]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the method as taught by Ofek into the combined method of DeFaria and Brent, the benefit of which is to guide the user to accomplish the task more proficiently (¶ [6]).
Claim 20, which is similar in scope to claim 10 as addressed above, is thus rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hau H. Nguyen whose telephone number is: 571-272-7787. The examiner can normally be reached on MON-FRI from 8:30-5:30.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard, can be reached on (571) 272-7773.
The fax number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/HAU H NGUYEN/Primary Examiner, Art Unit 2611
Prosecution Insights