DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Examiner’s Note
This Office Action is in response to application filed on 10/7/2024, where claims 1-20 are currently pending.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp. Claims 1, 8, and 15 have been provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. 11,256,405 (hereinafter the ‘405 Patent).
Although the claims at issue are not identical, they are not patentably distinct from each other because claim 1 of both the instant application and the ‘405 Patent are disclosing the method of capturing an image of an environment of a user that includes a controllable device, processing the image to determine the controllable device is included in the environment, determining one or more setting associated with the controllable device, and rendering element on a display corresponds to the one or more settings associated with the controllable device. Claim 1 of the instant application includes the additional limitation of determining classification for the controllable device. The US Patent No. 10,841,174 (Ely) teaches such.
The ‘405 Patent and Ely are analogous art to the claimed invention because they are concerning with interface for recognizing object(s) (i.e., same field of endeavor).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention having the ‘405 Patent and Ely before them to modify the method for controlling internet of things device of the ‘405 Patent to incorporate the function of determining classification for the controllable device by Ely. One of ordinary skill in the art would have combined the elements as claimed by known methods as disclosed by Ely (3:19-67), because the function of determining classification for the controllable device does not depend on method for controlling internet of things device. That is the function of determining classification for the controllable device performs the same function independent on which interface it is incorporated onto, and therefore, the result of the combination would have been predictable to one of ordinary skill in the art. The motivation to combine would have been to provide the correct information of the device as suggested by Ely (10:59-11:57).
Therefore, for the above reason(s), the claims are obvious variation of each other.
Although the claims at issue are not identical, they are not patentably distinct from each other because claim 8 of the instant application and claim 1 of the ‘405 Patent are disclosing capturing an image of an environment of a user that includes a controllable device, processing the image to determine the controllable device is included in the environment, determining one or more setting associated with the controllable device, and rendering element on a display corresponds to the one or more settings associated with the controllable device. Claim 8 of the instant claim is a device claim, and thus include the well-known elements to perform the function. Claim 8 of the instant application also includes the additional limitation of determining classification for the controllable device. The US Patent No. 10,841,174 (Ely) teaches such.
The ‘405 Patent and Ely are analogous art to the claimed invention because they are concerning with interface for recognizing object(s) (i.e., same field of endeavor).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention having the ‘405 Patent and Ely before them to modify the method for controlling internet of things device of the ‘405 Patent to incorporate the function of determining classification for the controllable device by Ely. One of ordinary skill in the art would have combined the elements as claimed by known methods as disclosed by Ely (3:19-67), because the function of determining classification for the controllable device does not depend on method for controlling internet of things device. That is the function of determining classification for the controllable device performs the same function independent on which interface it is incorporated onto, and therefore, the result of the combination would have been predictable to one of ordinary skill in the art. The motivation to combine would have been to provide the correct information of the device as suggested by Ely (10:59-11:57).
Therefore, for the above reason(s), the claims are obvious variation of each other.
Although the claims at issue are not identical, they are not patentably distinct from each other because claim 15 of the instant application and claim 1 of the ‘405 Patent are disclosing capturing an image of an environment of a user that includes a controllable device, processing the image to determine the controllable device is included in the environment, determining one or more setting associated with the controllable device, and rendering element on a display corresponds to the one or more settings associated with the controllable device. Claim 15 of the instant claim is a CRM claim, and thus include the well-known elements to store instructions and perform the function. Claim 15 of the instant application also includes the additional limitation of determining classification for the controllable device. The US Patent No. 10,841,174 (Ely) teaches such.
The ‘405 Patent and Ely are analogous art to the claimed invention because they are concerning with interface for recognizing object(s) (i.e., same field of endeavor).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention having the ‘405 Patent and Ely before them to modify the method for controlling internet of things device of the ‘405 Patent to incorporate the function of determining classification for the controllable device by Ely. One of ordinary skill in the art would have combined the elements as claimed by known methods as disclosed by Ely (3:19-67), because the function of determining classification for the controllable device does not depend on method for controlling internet of things device. That is the function of determining classification for the controllable device performs the same function independent on which interface it is incorporated onto, and therefore, the result of the combination would have been predictable to one of ordinary skill in the art. The motivation to combine would have been to provide the correct information of the device as suggested by Ely (10:59-11:57).
Therefore, for the above reason(s), the claims are obvious variation of each other.
This is a nonstatutory double patenting rejection.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-6, 8-13, and 15-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ely et al., (US 10,841,174 B1) (hereinafter Ely).
Referring to claim 1, Ely teaches a method implemented by one or more processors, the method comprising:
receiving image data that is based on an output of a camera of a user device that is directed at an environment (3:41-52, fig. 2, “a camera may gather images of external electronic devices in the user's surrounding environment”), wherein the user device includes an automated assistant interface that provides an interface between a user and an automated assistant (Examiner notes, according to 10:26-38, 10:54-58, and 11:58-12:21 of Ely described an interface for controlling one or more devices 14. It is implied an automated assistant is included, e.g., the software used to receive user control input and provide output to control the one or more devices 14 based on said input.);
processing the image data, wherein processing the image data comprises:
identifying, based on the image data, a controllable electronic device that is in the environment and that is captured in the image data (3:41-52, fig. 2, “a camera may gather images of external electronic devices in the user's surrounding environment, control circuitry may identify and/or gather information about the external electronic devices, a display system may display the camera images in real time”), and
determining a classification for the controllable electronic device that is included in the environment and that is captured in the image data (1:42-52, “The control circuitry may gather information about the external electronic devices in the environment, including determining a product type”);
determining, based on the classification, one or more settings associated with the controllable electronic device (15:36-57, fig. 3, “control circuitry 16 may display text on display system 24 in response to touch input to certain regions of an image or gesture input to certain regions of the environment. For example, if a user's finger 56 taps, presses, or otherwise provides touch input on keyboard 14’, text 66 may appear on display system 24. Text 66 may provide information about keyboard 14 (e.g., battery life, status of connections, capabilities, settings, etc.) and/or may provide a user with options for controlling keyboard 14 (e.g., turning keyboard 14 on, establishing or breaking a wireless connection between the keyboard and the computer monitor, adjusting settings on keyboard 14, etc.). Similarly, if a user's finger 56 taps, presses, or otherwise provides touch input to monitor 14’ text 66 may appear providing information about monitor 14 and/or providing options for controlling monitor 14.”); and
causing, based on determining one or more of the settings associated with the controllable electronic device, one or more selectable elements to be rendered at a display of the user device (13:50-14:4, “computer-generated display elements 48 may be used for receiving input from a user of device 10. After control circuitry 16 identifies and gathers information from devices 14, control circuitry 16 may assign corresponding user input functions to computer-generated display elements 48 and may overlay the computer-generated display elements 48 onto display system 24”), wherein each of the one or more selectable elements corresponds to one or more of the settings associated with the controllable electronic device (“In response to touch input or gesture input to computer-generated display elements 48, control circuitry 16 may take appropriate action. Illustrative input functions that may be performed by control circuitry 16 in response to touch input or gesture input on computer-generated display elements 48, touch input or gesture input to devices 14’ on display system 24, touch input or gesture input to other portions of display system 24, and/or any other suitable user input (e.g., motion input, audio input, touch or press input on other portions of device 10, etc.) include initiating or establishing a wireless communications link between devices 14, breaking a wireless communications link between devices 14, streaming or sharing content (e.g., audio, video, photos, a display screen, or other data) from one device (e.g., device 10 or 14) to another device (e.g., device 10 or 14), changing an output, operating state, or setting in devices 14”).
Referring to claim 2, Ely further teaches the method of claim 1, wherein identifying the controllable electronic device is further based on contextual data accessible to the user device (10:39-53, “The locations of devices 14’ on display system 24 may correspond to the locations of devices 14 in the real world (e.g., if device A is located to the left of device B in the real world, device A may be presented on display system 24 to the left of device B). Devices 14’ on display system 24 may include only devices 14 that are in the user's field of view, may include only devices 14 that are in the field of view of camera 20”)
Referring to claim 3, Ely further teaches the method of claim 2, wherein the contextual data includes room adjacency data that characterizes a layout of the environment (10:11-24, fig. 2, “control circuitry 16 may identify and gather information about external electronic devices in the surrounding environment and may present computer-generated images on display system 24. The computer-generated images may be overlaid onto images of the real world (e.g., captured by camera 20 and displayed in real time on display system 24), may be overlaid onto the real world itself (e.g., the real world as viewed through display system 24 in arrangements where display system 24 includes an optical combiner), or may be overlaid onto an entirely virtual world (e.g., an entirely computer-generated image on display system 24).” 10:39-53, fig. 2, “The environment presented on display system 24 may include devices 14’. Devices 14’ on display system 24 may be captured images of devices 14, may be actual real world devices 14 viewed through display system 24”. Examiner notes, the display system 24 is displaying the exact layout of the environment.)
Referring to claim 4, Ely further teaches the method of claim 2, wherein the contextual data characterizes a context of the user when the image data was captured (10:39-53, “The locations of devices 14’ on display system 24 may correspond to the locations of devices 14 in the real world (e.g., if device A is located to the left of device B in the real world, device A may be presented on display system 24 to the left of device B). Devices 14’ on display system 24 may include only devices 14 that are in the user's field of view”).
Referring to claim 5, Ely further teaches the method of claim 2, wherein the contextual data characterizes a context of the user device when the image data was captured (10:39-53, “The locations of devices 14’ on display system 24 may correspond to the locations of devices 14 in the real world (e.g., if device A is located to the left of device B in the real world, device A may be presented on display system 24 to the left of device B). Devices 14’ on display system 24…include only devices 14 that are in the field of view of camera 20”).
Referring to claim 6, Ely further teaches the method of claim 1, wherein determining the classification for the controllable electronic device that is included in the environment and that is captured in the image data is based on device data obtained from a manufacturer of the controllable electronic device (17:54-18:19, fig. 8, “Device 10 may use input-output devices 18 such as camera 20 or other light sensor to pick up visual features on external devices 14. Control circuitry 16 may process the captured images or other light sensor data to identify the type of device and/or gather other information about devices 14. For example, camera 20 may capture images of external devices 14, and control circuitry 16 may process the captured images and perform image recognition techniques to identify devices 14 (e.g., by identifying logo 68 on device 14, changes in contrast on device 14, colored or textured logos on device 14, corners or edges of device 14, product color, logo size relative to display or device size, etc.).” Examiner notes, some of the features used in identifying the type of device is produced by the manufacturer of the device, such as product color, corners or edges of device, etc.)
Regarding claims 8-13, these claims recite the user device that performs the steps of the method of claims 1-6 respectively; therefore, the same rationale of rejection is applicable.
Regarding claims 15-19, the instant claim recites the non-transitory computer readable storage medium storing instructions when executed, perform the steps of the method of claims 1-3, 5, and 6 respectively; therefore, the same rationale of rejection is applicable.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 7, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ely as applied to claims 1, 8, and 15 above, and in view of Dugan et al., (US 2006/0204107 A1) (hereinafter Dugan).
Referring to claim 7, Ely further teaches the method of claim 1, wherein determining the classification for the controllable electronic device that is included in the environment and that is captured in the image data (1:42-52, “The control circuitry may gather information about the external electronic devices in the environment, including determining a product type”). However, Ely does not explicitly teach the classification is based on device data obtained from a user...
Dugan teaches classification is based on device data obtained from a user...(¶ [0001], “The present invention relates generally to pattern recognition systems”. ¶ [0044], “The user can also specify "manual" features by selecting portions of the matching object. Therefore, various pattern matching scenarios can exist for a given matching object. For example, automatic features can be generated to search for a automobile of model jeep. The user can further select a location on a specific jeep that has a sticker or other unique identification mark.”)
Ely and Dugan are analogous art to the claimed invention because they are concerning with interface for recognizing object(s) (i.e., same field of endeavor).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention having Ely and Dugan before them to modify the intuitive electronic control interface of Ely to incorporate the function of manual feature selection for object recognition by Dugan. One of ordinary skill in the art would have combined the elements as claimed by known methods as disclosed by Dugan (¶ [0043]-[0049]), because the function of manual feature selection for object recognition does not depend on the intuitive electronic control interface. That is the function of manual feature selection for object recognition performs the same function independent on which interface it is incorporated onto, and therefore, the result of the combination would have been predictable to one of ordinary skill in the art. The motivation to combine would have been to reduce training time for object recognition as suggested by Dugan (¶ [0006]).
Regarding claim 14, the instant claim recites the user device that performs the steps of the method of claim 7; therefore, the same rationale of rejection is applicable.
Regarding claim 20, the instant claim recites the non-transitory computer readable storage medium storing instructions when executed, perform the steps of the method of claim 7; therefore, the same rationale of rejection is applicable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US 10,347,118 (Griffin) – discloses device for controlling plurality of target devices.
US 10,007,413 (Hill) – discloses interface for controlling remote objects.
US 8,830,267 (Brackney) – discloses augmented reality operation tool for remote controlling objects.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MONG-SHUNE CHUNG whose telephone number is (571) 270-5817. The examiner can normally be reached on M-F (9-5) EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott Baderman, can be reached at telephone number 571-272-3644. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR for authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/MONG-SHUNE CHUNG/
Primary Examiner, Art Unit 2118
Prosecution Insights