DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/27/2024 has been considered by the examiner.
Status of Claims
Claims 1-9 are currently pending in this application.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Objections
Claim 7 is objected to because of the following informalities:
Claim 7 line 6 recites “the fist registered image” this seems to be a typo. The Examiner suggests amending the limitation to recite “the first registered image”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-9 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, claim 1 lines 8-12 recite recording a captured image as ”a second registered image” when a difference is detected and recording a second captured image as “the second registered image” when a difference is not detected. It is unclear whether “a second registered image” recorded when a difference is detected is the same as “the second registered image” when a difference is not detected, or whether they are different images. For purposes of examination, the Examiner interprets “the second registered image” as a different image recorded after a difference is not detected. Independent claim 9 is similarly rejected.
Dependent claims 2-8 are rejected due to their dependency.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 7, 8, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Takayama et al. (US 11,430,415 B2) (hereinafter, “Takayama”) in view of Ohtsuka (US 9,036,056 B2).
Regarding claim 1, Takayama discloses an equipment operation device comprising: processing circuitry configured to (Column 9 [lines 57-61] "The management unit 100 includes: an appliance observing unit 101; a camera unit 102; an appliance information storing unit 103; an appliance identifying unit 104...")
acquire, from a camera, image data indicating a captured image in which operation target equipment and a peripheral region of the operation target equipment appear (Column 10 [lines 1-3] "The camera unit 102 captures images of the energy consuming appliances placed inside and outside of the home."; Column 10 [lines 10-15] "The appliance identifying unit 104 identifies the energy consuming appliance…included in the picture data item obtained by the camera unit 102."; Column 18 [lines 3-9] “The picture data item 217 with the superimposed presentation data items is of a captured image of a home space 500R in which the following are included: an energy consuming appliance (television receiver) 151; an energy consuming appliance (air conditioner) 152; and an energy consuming appliance (lighting equipment) 153.”; Examiner interprets the home space to include regions surrounding the appliances placed inside and outside the home);
perform a process of [detecting a difference] between the captured image indicated by the acquired image data (picture data item in Column 12 [lines 61-66] equates to image data) and [a first registered image] that has been recorded (Column 12 [lines 61-66] "The appliance identifying unit 104 performs image processing on the picture data item 212 to extract feature information. The appliance identifying unit 104 verifies whether the feature information matches the appliance identification information items 213 stored in the appliance information storing unit 103.");
[record the captured image as a second registered image that is different from the first registered image when the difference is detected, and acquire a second captured image differentiated from the first registered image that has been recorded and to record the second captured image as the second registered image that is different from the first registered image when the difference is not detected]; and
download an operation application of the operation target equipment appearing in the [second registered] image having been recorded (Column 48 [lines 18-21] "the management unit 100 includes an appliance content managing unit 2630 which manages the appliance content items downloaded from the appliance content server 2631."; Column 48 [lines 38-44] "The appliance content items include: a control module 2721 for causing the energy consuming appliances to operate in the new operation mode…").
However, Takayama fails to teach detecting a difference; record the captured image as a second registered image that is different from the first registered image when the difference is detected, and acquire a second captured image differentiated from the first registered image that has been recorded and to record the second captured image as the second registered image that is different from the first registered image when the difference is not detected.
Ohtsuka teaches detecting a difference (Column 8 [lines 51-55] “if there is no registered image that is the same as the distinguishing image for which the registration request is being made (NO at Step D10), the CPU 51 performs processing for generating); record the captured image (distinguishing image for which the registration request is being made in Column 8 [lines 51-55] equates to the captured image) as a second registered image (the new record registered as the end of the image database in Column 9 [lines 4-9] equates to a second registered image)that is different from the first registered image (registered image in Column 8 [lines 51-55] equates to the first registered image) when the difference is detected (Column 8 [lines 51-55] “if there is no registered image that is the same as the distinguishing image for which the registration request is being made (NO at Step D10), the CPU 51 performs processing for generating an accessory image based on this distinguishing image, prior to performing new registration”; Column 9 [lines 4-9] “When the accessory image is generated as described above, the CPU 51 issues a new ID, and after generating a new record composed of "ID", "distinguishing image", "accessory image", "accessory shape", "registration date", "date of use", and "status", adds and registers the new record at the end of the image database 7”), and acquire a second captured image (image acquired during re-registration in Column 13 [lines 14-18] equates to a second captured image) differentiated from the first registered image that has been recorded (Column 13 [lines 14-18] “When judged that the distinguishing image has already been registered (YES at Step E7) the mobile phone 1 displays a request to change the marker (Step E8) and returns to Step E2 to receive re-registration”, Examiner interprets that changing the marker to a different marker as the captured image being differentiated from the first registered image) and to record the second captured image (distinguishing image acquired from the mobile phone Column 8 [lines 23-28] equates to the second captured image) as the second registered image ("distinguishing images" in the image database in Column 8 [lines 23-28] equates to the second registered image) that is different from the first registered image when the difference is not detected (Column 8 [lines 23-28] “When one of the "distinguishing images" in the image database 7 is judged to match the distinguishing image acquired from the mobile phone 1 (YES at Step D3), the CPU 51 judges that an image that is the same as the "distinguishing image" for which the registration request is being made has already been registered”; Examiner interprets that the judged to match to be when the difference is not detected.).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Takayama’s reference to include detecting a difference; record the captured image as a second registered image that is different from the first registered image when the difference is detected, and acquire a second captured image differentiated from the first registered image that has been recorded and to record the second captured image as the second registered image that is different from the first registered image when the difference is not detected taught by Ohtsuka’s reference. The motivation for doing so would have been to prevent the duplicate generation of the same contents as suggested by Ohtsuka (see Ohtsuka, Column 11 [lines 49-52]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Ohtsuka with Takayama to obtain the invention specified in claim 1.
Regarding claim 5, which claim 1 is incorporated, Takayama discloses wherein the processing circuitry is further configured to receive selection of any one operation application from among one or more operation applications having been downloaded and execute the selected operation application (Column 48 [lines 34-37] "The appliance content selecting unit 2703 selects appliance content items to be applied to the energy consuming appliances 151 to 15n from among appliance content items stored in the appliance content storing unit 2701.").
Regarding claim 7, which claim 1 is incarnated, Takayama discloses wherein the processing circuitry is further configured to receive designation of a feature region that is a region of interest in the captured image indicated by the image data having been acquired when the processing circuitry performs the process [of detecting the difference] between the captured image indicated by the acquired image and [the fist registered image] that has been recorded (Column 10 [lines 10-15] "The appliance identifying unit 104 identifies the energy consuming appliance (specifically, the ID of the energy consuming appliance and the image position of the energy consuming appliance) included in the picture data item obtained by the camera unit 102."), and to output a captured image in which the feature region is designated (Column 10 [lines 34-36] "The output unit 108 outputs the picture data item with presentation data items superimposed by the smart information superimposing unit 107."; Column 17 [lines 34-40] "The superimposing unit 303 superimposes the presentation data items 302a generated by the presentation data generating unit 302 on the picture data item 215 with appliance identification information items input from the appliance identifying unit 104."; Column 17 [lines 41-45] "the presentation data items 302a corresponding to an identified energy consuming appliance is superimposed around the identified energy consuming appliance in the picture data item 215 with the appliance identification information items.").
However, Takayama fails to teach detecting a difference and the first registered image.
Ohtsuka teaches detecting a difference and the first registered image (registered image in Column 8 [lines 51-55] equates to the first registered image) (Column 8 [lines 51-55] “if there is no registered image that is the same as the distinguishing image for which the registration request is being made (NO at Step D10), the CPU 51 performs processing for generating an accessory image based on this distinguishing image, prior to performing new registration”; Column 9 [lines 4-9] “When the accessory image is generated as described above, the CPU 51 issues a new ID, and after generating a new record composed of "ID", "distinguishing image", "accessory image", "accessory shape", "registration date", "date of use", and "status", adds and registers the new record at the end of the image database 7”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Takayama’s reference to include detecting a difference and the first registered image taught by Ohtsuka’s reference. The motivation for doing so would have been to prevent the duplicate generation of the same contents as suggested by Ohtsuka (see Ohtsuka, Column 11 [lines 49-52]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Ohtsuka with Takayama to obtain the invention specified in claim 7.
Regarding claim 8, which claim 1 is incorporated, Takayama discloses wherein when receiving designation of a plurality of feature regions (Column 14 [lines 45-48] "the presentation need determining unit 301 counts the number N (N is an integer larger than or equal to 0) of energy consuming appliances identified in the picture data item 212"), the processing circuitry receives designation of a priority order of each of the feature regions, and outputs a captured image in which the feature region with the priority order is designated (Column 15 [lines 30-45] "M energy consuming appliances among the identified appliances are selected in descending order of the differences ΔE between the values in the demand plan and the current values...The process can be expressed as a process for selecting the M energy consuming appliances selected in descending order of presentation priority levels of the presentation data items and determining the presentation data items related to the energy consuming appliances as presentation data items which need to be presented.").
Regarding claim 9, Takayama discloses an equipment operation method comprising: acquiring from a camera, image data indicating a captured image in which operation target equipment and a peripheral region of the operation target equipment appear (Column 10 [lines 1-3] "The camera unit 102 captures images of the energy consuming appliances placed inside and outside of the home."; Column 10 [lines 10-15] "The appliance identifying unit 104 identifies the energy consuming appliance (specifically, the ID of the energy consuming appliance and the image position of the energy consuming appliance) included in the picture data item obtained by the camera unit 102."; Examiner interprets the home space to include regions surrounding the appliances placed inside and outside the home);
performing a process [of detecting a difference] between the captured image indicated by the acquired image data (picture data item in Column 12 [lines 61-66] equates to image data) and [a first registered image] that has been recorded (Column 12 [lines 61-66] "The appliance identifying unit 104 performs image processing on the picture data item 212 to extract feature information. The appliance identifying unit 104 verifies whether the feature information matches the appliance identification information items 213 stored in the appliance information storing unit 103. ");
[recording the captured image as a second registered image that is different from the first registered image when the difference is detected, and acquiring a second captured image differentiated from the first registered image that has been recorded, and recording the second captured image as the second registered image that is different from the first registered image when the difference is not detected]; and
downloading an operation application of the operation target equipment appearing in the second registered image having been recorded (Column 48 [lines 18-21] "the management unit 100 includes an appliance content managing unit 2630 which manages the appliance content items downloaded from the appliance content server 2631."; Column 48 [lines 38-44] "The appliance content items include: a control module 2721 for causing the energy consuming appliances to operate in the new operation mode…").
However, Takayama fails to teach detecting a difference; recording the captured image as a second registered image that is different from the first registered image when the difference is detected, and acquiring a second captured image differentiated from the first registered image that has been recorded, and recording the second captured image as the second registered image that is different from the first registered image when the difference is not detected.
Ohtsuka teaches detecting a difference (Column 8 [lines 51-55] “if there is no registered image that is the same as the distinguishing image for which the registration request is being made (NO at Step D10), the CPU 51 performs processing for generating); recording the captured image (distinguishing image for which the registration request is being made in Column 8 [lines 51-55] equates to the captured image) as a second registered image (the new record registered as the end of the image database in Column 9 [lines 4-9] equates to a second registered image) that is different from the first registered image (registered image in Column 8 [lines 51-55] equates to the first registered image) when the difference is detected (Column 8 [lines 51-55] “if there is no registered image that is the same as the distinguishing image for which the registration request is being made (NO at Step D10), the CPU 51 performs processing for generating an accessory image based on this distinguishing image, prior to performing new registration”; Column 9 [lines 4-9] “When the accessory image is generated as described above, the CPU 51 issues a new ID, and after generating a new record composed of "ID", "distinguishing image", "accessory image", "accessory shape", "registration date", "date of use", and "status", adds and registers the new record at the end of the image database 7”), and acquiring a second captured image (image acquired during re-registration in Column 13 [lines 14-18] equates to a second captured image) differentiated from the first registered image that has been recorded (Column 13 [lines 14-18] “When judged that the distinguishing image has already been registered (YES at Step E7) the mobile phone 1 displays a request to change the marker (Step E8) and returns to Step E2 to receive re-registration”, Examiner interprets that changing the marker to a different marker as the captured image being differentiated from the first registered image) and recording the second captured image (distinguishing image acquired from the mobile phone Column 8 [lines 23-28] equates to the second captured image) as the second registered image ("distinguishing images" in the image database in Column 8 [lines 23-28] equates to the second registered image) that is different from the first registered image when the difference is not detected (Column 8 [lines 23-28] “When one of the "distinguishing images" in the image database 7 is judged to match the distinguishing image acquired from the mobile phone 1 (YES at Step D3), the CPU 51 judges that an image that is the same as the "distinguishing image" for which the registration request is being made has already been registered”; Examiner interprets that the judged to match to be when the difference is not detected.).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Takayama’s reference to include detecting a difference; recording the captured image as a second registered image that is different from the first registered image when the difference is detected, and acquiring a second captured image differentiated from the first registered image that has been recorded, and recording the second captured image as the second registered image that is different from the first registered image when the difference is not detected taught by Ohtsuka’s reference. The motivation for doing so would have been to prevent the duplicate generation of the same contents as suggested by Ohtsuka (see Ohtsuka, Column 11 [lines 49-52]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Ohtsuka with Takayama to obtain the invention specified in claim 9.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Takayama et al. (US 11,430,415 B2) (hereinafter, “Takayama”) in view of Ohtsuka (US 9,036,056 B2), and further in view of Kim et al. (US 7,777,642 B2) (hereinafter, “Kim”).
Regarding claim 2, which claim 1 is incorporated, Takayama discloses wherein the processing circuitry acquires, from the camera, image data indicating a captured image in which the operation target equipment [with an identification marker] and the peripheral region appear as the second captured image (Column 10 [lines 1-3] "The camera unit 102 captures images of the energy consuming appliances placed inside and outside of the home."; Column 10 [lines 10-15] "The appliance identifying unit 104 identifies the energy consuming appliance…included in the picture data item obtained by the camera unit 102."; Column 18 [lines 3-9] “The picture data item 217 with the superimposed presentation data items is of a captured image of a home space 500R in which the following are included: an energy consuming appliance (television receiver) 151; an energy consuming appliance (air conditioner) 152; and an energy consuming appliance (lighting equipment) 153.”; Examiner interprets the home space to include regions surrounding the appliances placed inside and outside the home).
However, Takayama and Ohtsuka both fail to teach an identification marker.
Kim teaches an identification marker (Column 3 [lines 42-45] “images of the digital devices are captured by the infrared camera 102, and among them, the display information of unique tags attached to the images are displayed”; Column 4 [lines 1-5] “tags 202A, 202B, 202C, 202D, 202E and 204 of a bit pattern are attached onto a digital device (e.g., a TV set, an audio device and the like) 200 such as a home network of the present invention, the tags having location information and device information.”)
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Takayama in view of Ohtsuka to include an identification marker taught by Kim’s reference. The motivation for doing so would have been to identify the digital devices by the information tags as suggested by Kim (see Kim, Column 4 [lines 56-60]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Kim with Takayama and Ohtsuka to obtain the invention specified in claim 2
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Takayama et al. (US 11,430,415 B2) (hereinafter, “Takayama”) in view of Ohtsuka (US 9,036,056 B2), and further in view of Yamaguchi et al. (US 2012/0044402 A1) (hereinafter, “Yamaguchi”).
Regarding claim 6, which claim 1 is incorporated, Takayama and Ohtsuka both fail to teach wherein the processing circuitry is further configured to add a grid to the captured image indicated by the acquired image data and output the captured image with the grid.
Yamaguchi teaches wherein the processing circuitry is further configured to add a grid to the captured image indicated by the acquired image data and output the captured image with the grid (Paragraph [0052] “the display control unit 119 outputs image data in which the synthesized assistant image output from the assistant image synthesizing unit 117 is overlapped with a captured image”; Paragraph [0073] “the assistant image synthesizing unit 117 synthesizes the referential line image…The referential line image 210 b includes the display area 211 b that is grid lines dividing the screen 200 into four vertical parts and horizontal six parts…the assistant image synthesizing unit 117 synthesizes the other assistant image 225, as the highest-order layer, thereby completing the synthesized assistant image 250.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Takayama in view of Ohtsuka to include wherein the processing circuitry is further configured to add a grid to the captured image indicated by the acquired image data and output the captured image with the grid taught by Yamaguchi’s reference. The motivation for doing so would have been to assist users in determining an image capturing composition as suggested by Yamaguchi (see Yamaguchi, Paragraph [0051]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Yamaguchi with Takayama and Ohtsuka to obtain the invention specified in claim 6.
Allowable Subject Matter
Claims 3-4 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Claims 3-4 contain subject matter that is not disclosed or made obvious in the cited art.
In regards to claim 3, when considering claim 3 as a whole, prior art fails to disclose or render obvious, alone or in combination:
“[…] present guidance to prompt a user to paste the identification marker when the difference is not detected;
acquire image data indicating a captured image in which the operation target equipment to which the identification marker is pasted and the peripheral region appear from the camera after the guidance is presented; and
acquire, as the image data indicating the second captured image, the acquired image data after the guidance is presented.”
In regards to claim 4, when considering claim 4 as a whole, prior art fails to disclose or render obvious, alone or in combination:
“[…] combine an illustration component indicating an identification marker with the operation target equipment appearing in the captured image indicated by image data having been acquired, and acquire an image in which the operation target equipment with the identification marker and the peripheral region appear as the second captured image.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bae et al. (US 2018/0211093 A1) dividing input and registered images into regions and matching the images by comparing the corresponding regions.
Huang et al. (US 11,182,885 B2) discloses filtering an image, determining similarity degrees between the pixel points and a target region and adjusting filtering effects based on the similarity degrees.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UROOJ FATIMA whose telephone number is (571)272-2096. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/UROOJ FATIMA/Examiner, Art Unit 2676
/Henok Shiferaw/Supervisory Patent Examiner, Art Unit 2676
Prosecution Insights