DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement filed 02/11/2025 incorrectly listed US Patent Application Publication document number 8 whose citation was corrected by having the name and date changed to correspond to publication number information.
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Claims 1-19 have been interpreted under 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) to not invoke 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) claim interpretation.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-19 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-19 of U.S. Patent No. 12,190,580. Although the claims at issue are not identical, they are not patentably distinct from each other because pending method claims 1-19 are broader versions of system claims 1-19 and because pending method claims 1-19 only differ by being method claims rather than the patents system claims. It would have been obvious to one of ordinary sill in the art to draft method claim from system claims in order to broaden the claim coverage. Refer to the following table which correlates pending claims 1-19 filed on 12/24/2024 with patented system claims 1-19 in U.S. Patent No. 12,190,580 and to the following table which compares pending claims 1-19 filed on 12/24/2024 with patented system claims 1-19 in U.S. Patent No. 12,190,580.
Claims filed on 12/24/2024 correlated with claims of US Patent No. 12,190,580.
Claims filed on 12/24/2024
1
2
3
4
5
US Patent No. 12,190,580
1
2
3
4
5
Claims filed on 12/24/2024
6
7
8
9
10
US Patent No. 12,190,580
6
7
8
9
10
Claims filed on 12/24/2024
11
12
13
14
15
US Patent No. 12,190,580
11
12
13
14
15
Claims filed on 12/24/2024
16
17
18
19
US Patent No. 12,190,580
16
17
18
19
Claims filed on 12/24/2024 compared with US Patent No. 12,190,580.
Claims filed on 12/24/2024
1. A method for performing functions in augmented reality (AR) experiences, comprising:
responsive to processing an image of a monitored device having at least one object, determining a plurality of feature points;
storing a plurality of reference images with reference feature points and reference object zones, each reference object zone having a property associated therewith;
matching at least one of the plurality of feature points with at least one of the reference feature points from the plurality of reference images to compute a homography matrix;
transforming the image into a transformed image using the homography matrix;
identifying an object zone in the transformed image corresponding to a reference object zone of the reference object zones;
selecting a sub-image of the transformed image from the object zone, the sub-image selected based on the sub-image including one or more specified features of the at least one object determined from at least one of the reference images; and
determining an object state of the at least one object by processing the one or more specified features included in the sub-image based on the property associated with the reference object zone.
2. The method of claim 1, further comprising: obtaining the image from a mobile device.
3. The method of claim 1, wherein the image is a still picture or a frame of a video of the monitored device.
4. The method of claim 1, wherein the at least one object comprises at least one of a light emitting diode (LED) light, text, a switch, button, cable, or a potentially dangerous object.
5. The method of claim 4, wherein the image comprises a picture of at least one LED light, and the method further comprises: determining a state of the LED light by comparing the sub-image to a reference value.
6. The method of claim 5, wherein the state of the LED light is one of on, off, blinking, or associated with a particular color.
7. The method of claim 4, wherein the image comprises a picture of a button or switch, and the method further comprises: determining a state of the button or switch by comparing the sub-image to a reference value.
8. The method of claim 7, wherein the image comprises a picture of at least one cable or a picture of a cable receptacle, the method further comprises: determining a state of the cable or the cable receptacle by comparing the sub-image to a reference value.
9. The method of claim 4, wherein the image includes a picture of text, and the method further comprises: executing optical character recognition on the object zone of the transformed image to generate extracted text.
10. The method of claim 4, wherein
the image includes a picture of a potentially dangerous object, and the method further comprises:
determining a state of the potentially dangerous object in the object zone by comparing the picture to the reference feature points.
11. The method of claim 10, further comprising: responsive, to detecting a dangerous state, initiating a warning to be displayed at a user device regarding the potentially dangerous object.
12. The method of claim 1, further comprising: displaying, by a user device, the object state of the at least one object.
13. The method of claim 1, further comprising: providing troubleshooting instructions based on the object state of the at least one object.
14. The method of claim 1, wherein the image has a plurality of objects, and the method further comprises:
identifying a plurality of object zones in the transformed image, each associated with one of the plurality of objects, based on the reference object zones; and
determining an object state for each of the plurality of objects by processing the sub-image in each of the object zones of the transformed image by calling a function associated with a corresponding reference object zone of the reference object zones.
15. The method of claim 14, further comprising:
identifying a monitored device condition based on at least two object states from the object state determined for each of the plurality of objects.
16. The method of claim 14, wherein the computer is further configured to provide troubleshooting instructions based on a condition of the monitored device.
17. The method of claim 1, further comprising: accepting the image from a remote device; and transmitting the image to a computer that transforms the image using the homography matrix.
18. A method for performing functions in augmented reality (AR) experiences, comprising:
responsive to processing an image of a monitored device having at least one object, determining a plurality of feature points;
storing a plurality of reference images with reference feature points and reference object zones, each reference object zone having a property associated therewith;
matching at least some of the plurality of feature points with at least some of the
transforming the image into a transformed image using the homography matrix;
identifying an object zone in the transformed image corresponding to a reference object zone of the reference object zones, wherein the reference object zone is predefined based on a machine learning model trained to recognize specific objects and their states;
selecting a sub-image of the transformed image from the object zone, the sub-image selected based on the sub-image including one or more specified features of the at least one object determined from at least one of the reference images; and
determining an object state of the at least one object by processing the one or more specified features included in the sub-image based on the property associated with the reference object zone.
19. The method of claim 18, wherein determining the object state of the at least one object comprises:
comparing the one or more specified features with a set of predefined states in a database.
US Patent No. 12,190,580
1. A system for performing functions in augmented reality (AR) experiences, comprising:
a computer configured to receive an image of a monitored device having at least one object, and to process the image to determine a plurality of feature points; and
a database in data communication with the computer, the database configured to store a plurality of reference images with reference feature points and reference object zones, each reference object zone having a property associated therewith, wherein the computer is further configured to:
match at least some of the plurality of feature points with at least some of the reference feature points from the plurality of reference images to compute a homography matrix;
transform the image into a transformed image using the homography matrix;
identify an object zone in the transformed image corresponding to a reference object zone of the reference object zones;
select a sub-image of the transformed image from the object zone, the sub-image selected based on the sub-image including one or more specified features of the at least one object determined from at least one of the reference images; and
determine an object state of the at least one object by processing the one or more specified features included in the sub-image based on the property associated with the reference object zone.
2. The system of claim 1, wherein the image is provided to the computer by a mobile device.
3. The system of claim 1, wherein the image is a still picture or a frame of a video of the monitored device.
4. The system of claim 1, wherein the at least one object includes at least one of a LED light, text, a switch, button, cable, or a potentially dangerous object.
5. The system of claim 4, wherein: the image includes a picture of at least one LED light, and wherein the computer is further configured to determine a state of the LED light by comparing the sub-image to a reference value.
6. The system of claim 5, wherein the state of the LED light is one of on, off, blinking, or associated with a particular color.
7. The system of claim 4, wherein: the image includes a picture of a button or switch, and wherein the computer is further configured to determine a state of the button or switch by comparing the sub-image to a reference value.
8. The system of claim 7, wherein: the image includes a picture of at least one cable or a picture of a cable receptacle, and wherein the computer is further configured to determine a state of the cable or the cable receptacle by comparing the sub-image to a reference value.
9. The system of claim 4, wherein: the image includes a picture of text, and wherein the computer is further configured to execute optical character recognition on the object zone of the transformed image to generate extracted text.
10. The system of claim 4, wherein:
the image includes a picture of a potentially dangerous object, and wherein the computer is further configured to determine a state of the potentially dangerous object in the object zone by comparing the picture to the reference feature points.
11. The system of claim 10, wherein, if determined to be in a dangerous state, a user device is configured to display a warning regarding the potentially dangerous object.
12. The system of claim 1, wherein a user device is configured to display the object state of the at least one object.
13. The system of claim 1, the computer configured to provide troubleshooting instructions based on the object state of the at least one object.
14. The system of claim 1, wherein: the image has a plurality of objects, and wherein the computer is configured to:
identify a plurality of object zones in the transformed image, each associated with one of the plurality of objects, based on the reference object zones; and
determine an object state for each of the plurality of objects by processing the sub-image in each of the object zones of the transformed image by calling a function associated with a corresponding reference object zone of the reference object zones.
15. The system of claim 14, wherein the computer is further configured to
identify a monitored device condition based on at least two object states from the object state determined for each of the plurality of objects.
16. The system of claim 14, wherein the computer is further configured to provide troubleshooting instructions based on a condition of the monitored device.
17. The system of claim 1, further comprising a receiver that accepts the image from a remote device and transmits the image to the computer.
18. A system for performing functions in augmented reality (AR) experiences, comprising:
a computer configured to receive an image of a monitored device having at least one object, and to process the image to determine a plurality of feature points; and
a database in data communication with the computer, the database configured to store a plurality of reference images with reference feature points and reference object zones, each reference object zone having a property associated therewith, wherein the computer is further configured to:
match at least some of the plurality of feature points with at least some of the reference feature points from the plurality of reference images to compute a homography matrix;
transform the image into a transformed image using the homography matrix;
identify an object zone in the transformed image corresponding to a reference object zone of the reference object zones, wherein the reference object zone is predefined based on a machine learning model trained to recognize specific objects and their states;
select a sub-image of the transformed image from the object zone, the sub-image selected based on the sub-image including one or more specified features of the at least one object determined from at least one of the reference images; and
determine an object state of the at least one object by processing the one or more specified features included in the sub-image based on the property associated with the reference object zone.
19. The system of claim 18, wherein the computer is configured to determine the object state of the at least one object by:
comparing the one or more specified features with a set of predefined states in the database.
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Parangattil et al., US Patent Application Publication No. 2021/0091852 was cited in and discussed in parent US Patent Application No. 17/804,750.
Tathagata et al. US Patent Application Publication No. 2019/0372861, describes an augmented reality user interface for a server room to assist a user in servicing server room components.
Allowable Subject Matter
Claims 1-19 would be allowable if proper terminal disclaimer is filed addressing the nonstatutory double patenting rejection set forth in this Office action.
The following is a statement of reasons for the indication of allowable subject matter:
The reasons for allowance in parent US Patent Application No. 17/804,750, refer to Applicant’s response filed on 07/11/2024 and refer to Notice of Allowance mailed on 09/03/2024, for the system claims of the parent application applies to the method claim of this child application and applies to the prior of record in this child application.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFERY A BRIER whose telephone number is (571)272-7656. The examiner can normally be reached on Mon-Fri from 8:30am-3:00pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao M Wu, can be reached at telephone number 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
JEFFERY A. BRIER
Primary Examiner
Art Unit 2613
/JEFFERY A BRIER/Primary Examiner, Art Unit 2613
Prosecution Insights