DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Amendment
The preliminary amendment submitted on 04/25/2025 is acknowledged. Claims 1-9 are pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted is considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Handa (US Publication Number 2022/0139071 A1) in view of Hanzawa et al. (US Publication Number 2019/0139212 A1, hereinafter “Hanzawa”).
(1) regarding claim 1:
As explained in fig. 1, Handa disclosed an image processing device (para. [0013], note that an information processing device 1 according to the present disclosure recognizes and judges objects from an image using recognizer that is machine-trained through one-shot learning using a Siamese Network) comprising:
a learning part that learns an image by using deep learning (para. [0015], note that computational graphs (functions) used in machine learning are generally called models);
an acquisition part that acquires a first feature amount of each of a first image and a second image each extracted in a first layer in the deep learning (para. [0018], note that in a model, layers close to the input mainly extract feature amounts of the input data. Such layers that are close to the input use many sum-of-products operations to determine data correlation. Also see para. [0026], note that the two image feature amount extraction layers are trained by inputting combination data of vehicles and non-vehicles (step S2). For example, in the present disclosure, image data of an image of a vehicle is first input into one of the image feature amount extraction layers) and a second feature amount of each of a first image and a second image each extracted in a second layer different from the first layer in the deep learning (para. [0026], note that image data of an image of an object other than a vehicle (such as a person or a landscape) is input into the other image feature amount extraction layer, and the differential layer is then caused to detect differences in the feature amounts between the two images).
Handa disclosed most of the subject matter as described as above except for specifically teaching a display controller that superimposes and displays first feature amount information on the first feature amount and the second feature amount extracted from the first image extracted in the first layer on the first image extracted in the first layer, and superimposes and displays second feature amount information on the first feature amount and the second feature amount extracted from the second image extracted in the second layer on the second image extracted in the second layer.
However, Hanzawa disclosed a display controller that superimposes and displays first feature amount information on the first feature amount and the second feature amount extracted from the first image extracted in the first layer on the first image extracted in the first layer (para. [0106], note that he display unit may display an image and an image in which a partial image has been emphasized in a superimposed manner, or may display an image and identification information of the object to be inspected), and superimposes and displays second feature amount information on the first feature amount and the second feature amount extracted from the second image extracted in the second layer on the second image extracted in the second layer (para. [0142], note that an example of a superimposed image IM3 is illustrated in which the image IM2 in which a partial image is emphasized is subjected to transparency processing is superimposed with the measurement image IM1 of the object to be inspected shown in FIG. 6. As a result of displaying such a superimposed image IM3 as well, the region of the measurement image based on which the identification device has determined that a defect is included can be easily confirmed).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach a display controller that superimposes and displays first feature amount information on the first feature amount and the second feature amount extracted from the first image extracted in the first layer on the first image extracted in the first layer, and superimposes and displays second feature amount information on the first feature amount and the second feature amount extracted from the second image extracted in the second layer on the second image extracted in the second layer. The suggestion/motivation for doing so would have been in order to add learning data and increasing variations thereof in order to improve the identification accuracy of the identification device such that whether or not an object to be inspected includes a defect can be identified with high accuracy (para. [0008]). Therefore, it would have been obvious to combine Handa with Hanzawa to obtain the invention as specified in claim 1.
(2) regarding claim 2:
Handa further disclosed the image processing device according to Claim 1, further comprising a reception part that receives necessity of relearning using at least one of the first image and the second image in the learning part (para. [0033], note that the parameters of the image feature amount extraction layers are adjusted through training which reduces differences between the detected feature amounts when image data of an image of a motorcycle is input to the two image feature amount extraction layers, and increases differences in other cases. As a result, motorcycle recognition parameters 62, which are suited to judging whether an object in an image is a motorcycle or not and which are shared by the two image feature amount extraction layer).
(3) regarding claim 3:
Handa further disclosed the image processing device according to Claim 2, wherein the first image is an image learned by the learning part (para. [0028], note that image data of an image of a vehicle is input into the two image feature amount extraction layers), the second image is an image that has not been learned by the learning part (para. [0027], note that mage data of an image of an object other than a vehicle (such as a person or a landscape) is input into the one image feature amount extraction layer), and the learning part performs relearning using the first feature amount information and the second image when the reception part receives an instruction requiring relearning (para. [0029], note that the parameters of the image feature amount extraction layers are adjusted through training which reduces differences between the detected feature amounts when image data of an image of a vehicle is input to the two image feature amount extraction layers, and increases differences in other cases. As a result, vehicle recognition parameters 61, which are suited to judging whether an object in an image is a vehicle or not and which are shared by the two image feature amount extraction layers, are obtained).
(4) regarding claim 4:
Handa further disclosed the image processing device according to Claim 1, further comprising a calculator that calculates similarity between the first feature amount information and the second feature amount information, wherein the learning part determines necessity of relearning using at least one of the first image and the second image on a basis of the similarity (para. [0055], note that the information processing device 1 judges whether an object in a captured image is a vehicle or a non-vehicle, or a motorcycle or a non-motorcycle, on the basis of feature amounts in the image data of the captured image and the similarity (resemblance) to the vehicle image reference data 51 or the motorcycle image reference data 52).
The proposed rejection of claim 1 renders obvious the steps of the method of claim 9 because these steps occur in the operation of the proposed rejection as discussed above. Thus, the arguments similar to that presented above for claim 1 is equally applicable to claim 9.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Handa and Hanzawa, further in view of Assaf et al. (US Publication Number 2018/0165518 A1, hereinafter “Assaf”).
(1) regarding claim 5:
Handa further disclosed the image processing device according to Claim 1, wherein the first image is an image learned by the learning part (para. [0076], note that the motorcycle recognition parameters 62, which are examples of parameters learned through machine learning), the second image is an image that has not been learned by the learning part and is an image generated by an imaging part photographing a target object (para. [0026], note that mage data of an image of an object other than a vehicle (such as a person or a landscape) is input into the other image feature amount extraction layer. Also see para. [0039]).
Handa disclosed most of the subject matter as described as above except for specifically teaching the image processing device further includes an output part that outputs control information related to control of operation of a robot that performs predetermined processing on the target object on a basis of the second image.
However, Assaf disclosed the image processing device further includes an output part that outputs control information related to control of operation of a robot that performs predetermined processing on the target object on a basis of the second image (para. [0021], note that the robot can navigate itself about an area, e.g., a house, office, or warehouse, to explore the area and to recognize objects and/or individual locations (e.g., rooms) of the area. The objects can include physical objects, people, animals, plants, and/or other appropriate types of objects. If the robot cannot identify a particular object or location, the robot can store data for the unidentified item, e.g., a location and image, locally or at a remote server system).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach the image processing device further includes an output part that outputs control information related to control of operation of a robot that performs predetermined processing on the target object on a basis of the second image. The suggestion/motivation for doing so would have been in order to improve object recognition for robots (abs.). Therefore, it would have been obvious to combine Handa and Hanzawa with Assaf to obtain the invention as specified in claim 5.
Claim(s) 6-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Handa and Hanzawa, further in view of Kitazawa (US Publication Number 2022/0392107 A1).
(1) regarding claim 6:
Handa disclosed most of the subject matter as described as above except for specifically teaching wherein the acquisition part acquires the first feature amount and the second feature amount of each of a plurality of the second images in which target objects having different sizes are shown, and the display controller causes each of the plurality of second images to display the plurality of second images in which the second feature amount information is superimposed and displayed in order of one of the sizes of the target objects appearing in the second image.
However, Kitazawa disclosed wherein the acquisition part acquires the first feature amount and the second feature amount of each of a plurality of the second images in which target objects having different sizes are shown (para. [0073], note that the feature amount extraction unit 130 converts (resizes, or the like) the images 1005 to 1007 into images of a defined image size (here, 32-pixels×32-pixels)), and the display controller causes each of the plurality of second images to display the plurality of second images in which the second feature amount information is superimposed and displayed in order of one of the sizes of the target objects appearing in the second image (para. [0031], note that fig. 4 illustrates a state in which a first feature map is superimposed on a first image having a size of a 640-pixel width and a 480-pixel height, the first feature map having registered therein a feature amount of each divided region of the first image when the first image is divided into 24 divided regions horizontally and 16 divided regions vertically).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the acquisition part acquires the first feature amount and the second feature amount of each of a plurality of the second images in which target objects having different sizes are shown, and the display controller causes each of the plurality of second images to display the plurality of second images in which the second feature amount information is superimposed and displayed in order of one of the sizes of the target objects appearing in the second image. The suggestion/motivation for doing so would have been in order to provides a technique for correctly discriminating from other objects and tracking a tracking target object, even in the presence of objects resembling the tracking object in appearance (para. [0004]). Therefore, it would have been obvious to combine Handa and Hanzawa with Kitazawa to obtain the invention as specified in claim 6.
(2) regarding claim 7:
Handa disclosed most of the subject matter as described as above except for specifically teaching wherein at least one of the first image and the second image is an image that has not been learned by the learning part and is an image generated by an imaging part photographing a target object, and the image processing device further includes a distance controller that controls a distance between the imaging part and the target object.
However, Kitazawa disclosed wherein at least one of the first image and the second image is an image that has not been learned by the learning part and is an image generated by an imaging part photographing a target object (para. [0027], note that the first image and the second image are respectively a still image of interest among a plurality of still images captured regularly or irregularly, and a still image captured after the still image of interest), and the image processing device further includes a distance controller that controls a distance between the imaging part and the target object (para. [0044], note that the feature extraction unit 130 calculates the distance between feature amounts d.sub.1 between a feature amount at a corresponding position on the first feature map corresponding to the “position of the tracking target object 501 in the first image” indicated by the first correct-answer data).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein at least one of the first image and the second image is an image that has not been learned by the learning part and is an image generated by an imaging part photographing a target object, and the image processing device further includes a distance controller that controls a distance between the imaging part and the target object. The suggestion/motivation for doing so would have been in order to provide a technique for correctly discriminating from other objects and tracking a tracking target object, even in the presence of objects resembling the tracking object in appearance (para. [0004]). Therefore, it would have been obvious to combine Handa and Hanzawa with Kitazawa to obtain the invention as specified in claim 7.
(3) regarding claim 8:
Handa disclosed most of the subject matter as described as above except for specifically teaching wherein the first feature amount information is information indicating a position of each of the first feature amount and the second feature amount in the first image, and the second feature amount information is information indicating a position of each of the first feature amount and the second feature amount in the second image.
However, Kitazawa disclosed wherein the first feature amount information is information indicating a position of each of the first feature amount and the second feature amount in the first image (para. [0025], note that the region information indicating an image region of the tracking target object is information that includes, for example in a case where the image region is a rectangular region, the center position of the rectangular region (position of the tracking target object)), and the second feature amount information is information indicating a position of each of the first feature amount and the second feature amount in the second image (para. [0030], note that the feature extraction unit 130 then identifies a corresponding position on the second feature map corresponding to the center position of the image region of the tracking target object indicated by the second correct-answer data, and acquires the feature amount at the identified corresponding position as “the feature amount of the tracking target object”).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the first feature amount information is information indicating a position of each of the first feature amount and the second feature amount in the first image, and the second feature amount information is information indicating a position of each of the first feature amount and the second feature amount in the second image. The suggestion/motivation for doing so would have been in order to provide a technique for correctly discriminating from other objects and tracking a tracking target object, even in the presence of objects resembling the tracking object in appearance (para. [0004]). Therefore, it would have been obvious to combine Handa and Hanzawa with Kitazawa to obtain the invention as specified in claim 8.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kim et al. (US Patent Number 11,989,957 B2) disclosed an apparatus for controlling object tracking and a method therefor are provided. The apparatus includes an object detector configured to detect an object in an image, an object tracker configured to track the object, a learning device configured to learn whether to enable the object detector based on features of the object and tracking results of the object tracker, and a controller configured to determine whether to enable the object detector by interworking with the learning device.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Hilina K Demeter whose telephone number is (571) 270-1676.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon could be reached at (571) 270- 0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about PAIR system, see http://pari-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HILINA K DEMETER/Primary Examiner, Art Unit 2617
Prosecution Insights