DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on January 15, 2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 4 and 10 are objected to because of the following informalities:
Claim 4, “optional viewpoints” is believed to be a potential translation error. Potentially the claim should read “multiple viewpoints” or “different viewpoints”
Claim 10, “a video creation circuitry” should read “video creation circuitry”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Invoked despite absence of “means”
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“object information generation device” in claim 10
“a search assistance device” in claim 10
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Based on the 35 USC 112(f) interpretation as noted above, corresponding 35 USC § 112(b) and 35 USC § 112(a) rejections were considered. Upon review of the specification, it was determined USC § 112(b) and 35 USC § 112(a) rejections were unnecessary. See below for the relevant PG Publication paragraphs:
Element
Element Number
Structure/Algorithm (when needed)
Object information generation device
216 (see paragraph 0033)
Structure: paragraph 0033-0034, “The object information generation device 216 is, for example, an infrared camera device 212 and a radar device 214.”… “Further, in the present embodiment, the object information generation device 216 includes the infrared camera device 212 and the radar device 214 but is not limited thereto and may have another configuration as long as it is capable of generating object information as described below. The object information generation device 216 acquires object information indicating an object included in the captured video. The infrared camera device 212 generates an infrared video as object information by emitting infrared light. The radar device 214 irradiates a search area with pulses of laser light and measures the reflected light to measure distance information to an object as object information.”
Search assistance device
100 (see paragraph 0047)
Structure: paragraph 0031, “The search assistance device 100 performs processing to assist the search of a search target object. The search assistance device 100 may be configured, for example, to be connected to a plurality of camera systems 200 and a plurality of searcher's terminal devices 300 and to provide a cloud service that assists the search of a search target object.” Paragraph 0038, “Each unit, such as the captured video acquisition unit 106, the object information acquisition unit 108, the setting unit 110, the conversion unit 112, the determination unit 114, the video creation unit 116, and the search unit 118, is realized by a computer, such as a CPU mounted on the search assistance device 100, executing a program stored in a program memory.”
Algorithm: paragraph 0047, “FIG. 4 is a flowchart showing an example of a processing procedure in the search assistance device 100 according to the first embodiment.” See rest of paragraph for explicit algorithm details
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1- 3 and 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent No. 10,726,312 to Butt et al. (hereinafter Butt), and further in view of U.S. Patent No. 9,904,852 to Divakaran et al. (hereinafter Divakaran).
Regarding independent claim 1, Butt discloses A search assistance device (abstract, “There is provided an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects.”) comprising:
captured video acquisition circuitry configured to acquire a captured video (abstract, “an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects.” See also Figure 4, element 108; column 8, line 54, “The video capture and playback system 100 includes at least one video capture device 108 being operable to capture a plurality of images and produce image data representing the plurality of captured images. The video capture device 108 or camera 108 is an image capturing device and includes security video cameras.”);
setting circuitry configured to set a search target image including a search target object (column 17, line 38, “In the first method of initiating Appearance Search 500, an image of an object of interest is received 502 at the client 420 where it is sent to the Process 408 to generate 504 a feature vector of the object of interest.”);
object information acquisition circuitry configured to acquire object information indicating an object included in the captured video (column 11, line 22, “The video analytics module 224 receives image data and analyzes the image data to determine properties or characteristics of the captured image or video and/or of objects found in the scene represented by the image or video.”);
video creation circuitry configured to create a comparison video to be compared to search for the search target object using an image that shows the peripheral object in the captured video in a case where the determination circuitry determines that part of the search target candidate is hidden by the peripheral object (column 13, line 49, “In a further embodiment, the size of the extracted image is larger than the actual boundaries of the object that has been detected, herein called a Padded cropped bounding box (also referred to as a “Padded Chip”). The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object. For greater clarity, Padded cropped bounding boxes have larger images then cropped bounding boxes of images of objects within bounding boxes (herein called non-Padded cropped bounding boxes);” column 14, line 11, “In some embodiments, the cropped bounding boxes, including the Padded cropped bounding boxes and the non-Padded cropped bounding boxes, may be reference coordinates of image frames of the video instead of actual extracted images of image frames of the video.”).
Butt fails to explicitly disclose as further recited. However, Divakaran discloses determination circuitry configured to determine whether or not part of a search target candidate is hidden by a peripheral object using the search target image or the object information (Figure 9, specifically elements 910, 916, 918, 920, and 924-926; claim 1: identifying one or more occlusions depicted in the video stream; inferring a spatial relationship between at least two of the objects in the group of objects based on the detected parts and the identified occlusions)
Butt is directed toward, “There is provided an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects (abstract).” Divakaran is directed toward, “A system for object detection and tracking includes technologies to, among other things, detect and track moving objects, such as pedestrians and/or vehicles, in a real-world environment, handle static and dynamic occlusions, and continue tracking moving objects across the fields of view of multiple different cameras (abstract).” AS can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention Butt and Divakaran are directed toward similar methods of endeavor of object detection. Further, one of of ordinary skill in the art and exemplified in Butt would easily understand objects often overlap in video and image data making analysis more difficult (column 13, line 52, “The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object.”). When interested in an object for either detecting or tracking purposes, occluding objects can cause errors. Thus, in order to get the most accurate object analysis result, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Divakaran in order to ensure the relationship between objects that may be obstructed.
Regarding dependent claim 2, the rejection of claim 1 is incorporated herein. Additionally, Butt in the combination further discloses wherein the video creation circuitry is configured to create an entire video representing an entirety of the search target candidate as the comparison video (segmenting the object across multiple frames it is in is read as the entire video; column 13, line 49, “In a further embodiment, the size of the extracted image is larger than the actual boundaries of the object that has been detected, herein called a Padded cropped bounding box (also referred to as a “Padded Chip”). The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object. For greater clarity, Padded cropped bounding boxes have larger images then cropped bounding boxes of images of objects within bounding boxes (herein called non-Padded cropped bounding boxes);” column 14, line 11, “In some embodiments, the cropped bounding boxes, including the Padded cropped bounding boxes and the non-Padded cropped bounding boxes, may be reference coordinates of image frames of the video instead of actual extracted images of image frames of the video.”).
Regarding dependent claim 3, the rejection of claim 2 is incorporated herein. Additionally, Butt discloses wherein the object information acquisition circuitry is an infrared camera or a radar device (column 9, line 5, “In other examples, the at least one image sensor 116 may be operable to capture light outside the visible light range, such as in the infrared and/or ultraviolet range. ”), and
wherein the video creation circuitry is configured to create the entire video using an infrared camera image or radar information acquired by the infrared camera or the radar device (column 9, line 5, “In other examples, the at least one image sensor 116 may be operable to capture light outside the visible light range, such as in the infrared and/or ultraviolet range.” If the data obtained is infrared, the downstream processing is read to be acting on the infrared data to generate the video).
Regarding independent claim 8, the rejection of claim 1 applies directly. Additionally, Butt further discloses A search assistance method (abstract, “There is provided an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects. The system comprises one or more processors and memory comprising computer program code stored on the memory and configured when executed by the one or more processors to cause the one or more processors to perform a method. ”) comprising:
setting a search target image including a search target object (column 17, line 38, “In the first method of initiating Appearance Search 500, an image of an object of interest is received 502 at the client 420 where it is sent to the Process 408 to generate 504 a feature vector of the object of interest.”);
acquiring a captured video and object information indicating an object included in the captured video (abstract, “an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects.” See also Figure 4, element 108; column 8, line 54, “The video capture and playback system 100 includes at least one video capture device 108 being operable to capture a plurality of images and produce image data representing the plurality of captured images. The video capture device 108 or camera 108 is an image capturing device and includes security video cameras.” column 11, line 22, “The video analytics module 224 receives image data and analyzes the image data to determine properties or characteristics of the captured image or video and/or of objects found in the scene represented by the image or video.”);
creating a comparison video to be compared to search for the search target object using an image that shows the peripheral object in the captured video in a case where it is determined that part of the search target candidate is hidden by the peripheral object (column 13, line 49, “In a further embodiment, the size of the extracted image is larger than the actual boundaries of the object that has been detected, herein called a Padded cropped bounding box (also referred to as a “Padded Chip”). The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object. For greater clarity, Padded cropped bounding boxes have larger images then cropped bounding boxes of images of objects within bounding boxes (herein called non-Padded cropped bounding boxes);” column 14, line 11, “In some embodiments, the cropped bounding boxes, including the Padded cropped bounding boxes and the non-Padded cropped bounding boxes, may be reference coordinates of image frames of the video instead of actual extracted images of image frames of the video.”).
Butt fails to explicitly disclose as further recited. However, Divakaran discloses determining whether or not part of a search target candidate is hidden by a peripheral object using the search target image or the object information (Figure 9, specifically elements 910, 916, 918, 920, and 924-926; claim 1: identifying one or more occlusions depicted in the video stream; inferring a spatial relationship between at least two of the objects in the group of objects based on the detected parts and the identified occlusions).
Butt is directed toward, “There is provided an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects (abstract).” Divakaran is directed toward, “A system for object detection and tracking includes technologies to, among other things, detect and track moving objects, such as pedestrians and/or vehicles, in a real-world environment, handle static and dynamic occlusions, and continue tracking moving objects across the fields of view of multiple different cameras (abstract).” AS can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention Butt and Divakaran are directed toward similar methods of endeavor of object detection. Further, one of of ordinary skill in the art and exemplified in Butt would easily understand objects often overlap in video and image data making analysis more difficult (column 13, line 52, “The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object.”). When interested in an object for either detecting or tracking purposes, occluding objects can cause errors. Thus, in order to get the most accurate object analysis result, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Divakaran in order to ensure the relationship between objects that may be obstructed.
Regarding independent claim 9, the rejection of claim 1 applies directly. Additionally, Butt further discloses A computer readable non-transitory recording medium having a program causing a computer mounted on a search assistance device (abstract, “There is provided an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects. The system comprises one or more processors and memory comprising computer program code stored on the memory and configured when executed by the one or more processors to cause the one or more processors to perform a method. ”) to execute:
setting a search target image including a search target object (column 17, line 38, “In the first method of initiating Appearance Search 500, an image of an object of interest is received 502 at the client 420 where it is sent to the Process 408 to generate 504 a feature vector of the object of interest.”);
acquiring a captured video and object information indicating an object included in the captured video (abstract, “an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects.” See also Figure 4, element 108; column 8, line 54, “The video capture and playback system 100 includes at least one video capture device 108 being operable to capture a plurality of images and produce image data representing the plurality of captured images. The video capture device 108 or camera 108 is an image capturing device and includes security video cameras.” column 11, line 22, “The video analytics module 224 receives image data and analyzes the image data to determine properties or characteristics of the captured image or video and/or of objects found in the scene represented by the image or video.”);
creating a comparison video to be compared to search for the search target object using an image that shows the peripheral object in the captured video in a case where it is determined that part of the search target candidate is hidden by the peripheral object (column 13, line 49, “In a further embodiment, the size of the extracted image is larger than the actual boundaries of the object that has been detected, herein called a Padded cropped bounding box (also referred to as a “Padded Chip”). The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object. For greater clarity, Padded cropped bounding boxes have larger images then cropped bounding boxes of images of objects within bounding boxes (herein called non-Padded cropped bounding boxes);” column 14, line 11, “In some embodiments, the cropped bounding boxes, including the Padded cropped bounding boxes and the non-Padded cropped bounding boxes, may be reference coordinates of image frames of the video instead of actual extracted images of image frames of the video.”).
Butt fails to explicitly disclose as further recited. However, Divakaran discloses determining whether or not part of a search target candidate is hidden by a peripheral object using the search target image or the object information (Figure 9, specifically elements 910, 916, 918, 920, and 924-926; claim 1: identifying one or more occlusions depicted in the video stream; inferring a spatial relationship between at least two of the objects in the group of objects based on the detected parts and the identified occlusions).
Butt is directed toward, “There is provided an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects (abstract).” Divakaran is directed toward, “A system for object detection and tracking includes technologies to, among other things, detect and track moving objects, such as pedestrians and/or vehicles, in a real-world environment, handle static and dynamic occlusions, and continue tracking moving objects across the fields of view of multiple different cameras (abstract).” AS can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention Butt and Divakaran are directed toward similar methods of endeavor of object detection. Further, one of ordinary skill in the art and exemplified in Butt would easily understand objects often overlap in video and image data making analysis more difficult (column 13, line 52, “The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object.”). When interested in an object for either detecting or tracking purposes, occluding objects can cause errors. Thus, in order to get the most accurate object analysis result, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Divakaran in order to ensure the relationship between objects that may be obstructed.
Regarding independent claim 10, the rejection of claim 1 applies directly. Additionally, Butt further discloses A search assistance system (abstract, “There is provided an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects. The system comprises one or more processors and memory comprising computer program code stored on the memory and configured when executed by the one or more processors to cause the one or more processors to perform a method. ”) comprising:
an imaging device configured to generate a captured (abstract, “an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects.” See also Figure 4, element 108; column 8, line 54, “The video capture and playback system 100 includes at least one video capture device 108 being operable to capture a plurality of images and produce image data representing the plurality of captured images. The video capture device 108 or camera 108 is an image capturing device and includes security video cameras.”);
an object information generation device configured to generate object information indicating an object included in the captured video (column 11, line 22, “The video analytics module 224 receives image data and analyzes the image data to determine properties or characteristics of the captured image or video and/or of objects found in the scene represented by the image or video.”); and
a search assistance device including setting circuitry configured to set a search target image including a search target object (column 17, line 38, “In the first method of initiating Appearance Search 500, an image of an object of interest is received 502 at the client 420 where it is sent to the Process 408 to generate 504 a feature vector of the object of interest.”),
a video creation circuitry configured to create a comparison video to be compared to search for the search target object using an image that shows the peripheral object in the captured video in a case where the determination circuitry determines that part of the search target candidate is hidden by the peripheral object (column 13, line 49, “In a further embodiment, the size of the extracted image is larger than the actual boundaries of the object that has been detected, herein called a Padded cropped bounding box (also referred to as a “Padded Chip”). The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object. For greater clarity, Padded cropped bounding boxes have larger images then cropped bounding boxes of images of objects within bounding boxes (herein called non-Padded cropped bounding boxes);” column 14, line 11, “In some embodiments, the cropped bounding boxes, including the Padded cropped bounding boxes and the non-Padded cropped bounding boxes, may be reference coordinates of image frames of the video instead of actual extracted images of image frames of the video.”).
Butt fails to explicitly disclose as further recited. However, Divakaran discloses determination circuitry configured to determine whether or not part of a search target candidate is hidden by a peripheral object using the search target image or the object information (Figure 9, specifically elements 910, 916, 918, 920, and 924-926; claim 1: identifying one or more occlusions depicted in the video stream; inferring a spatial relationship between at least two of the objects in the group of objects based on the detected parts and the identified occlusions)
Butt is directed toward, “There is provided an appearance search system comprising one or more cameras configured to capture video of a scene, the video having images of objects (abstract).” Divakaran is directed toward, “A system for object detection and tracking includes technologies to, among other things, detect and track moving objects, such as pedestrians and/or vehicles, in a real-world environment, handle static and dynamic occlusions, and continue tracking moving objects across the fields of view of multiple different cameras (abstract).” AS can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention Butt and Divakaran are directed toward similar methods of endeavor of object detection. Further, one of ordinary skill in the art and exemplified in Butt would easily understand objects often overlap in video and image data making analysis more difficult (column 13, line 52, “The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object.”). When interested in an object for either detecting or tracking purposes, occluding objects can cause errors. Thus, in order to get the most accurate object analysis result, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Divakaran in order to ensure the relationship between objects that may be obstructed.
Claim(s) 4-5 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Butt further in view of Divakaran as applied to claim 1 above, and further in view of U.S. Publication No. 2010/0165105 to Mizusawa (hereinafter Mizusawa).
Regarding dependent claim 4, the rejection of claim 1 is incorporated herein. Additionally, Butt and Divakaran in the combination fail to explicitly disclose comprising:
conversion circuitry configured to convert the search target image set by the setting circuitry into converted images viewed from optional viewpoints.
However, Mizusawa discloses further comprising:
conversion circuitry configured to convert the search target image set by the setting circuitry into converted images viewed from optional viewpoints (abstract, “An image synthesis section references the mapping table storage section and converts an image input from an image pickup section into a virtual image viewed from a predetermined virtual eye point.”).
As noted above, Butt and Divakaran are directed toward similar methods of endeavor of object analysis in video data. Further, Mizusawa is directed toward, “image synthesis section references the mapping table storage section and converts an image input from an image pickup section into a virtual image viewed from a predetermined virtual eye point (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Butt, Divakaran and Mizusawa are directed toward similar methods of endeavor of object analysis. Further, Butt and Divakaran allow for performing object detection. One of ordinary skill in the art would easily know that often objects are obstructed making analysis difficult. However, the object may not be occluded from all viewpoints. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Mizusawa in order to generate views of an object that are not occluded, allowing for more simple and accurate downstream analysis.
Regarding dependent claim 5, the rejection of claim 4 is incorporated herein. Additionally, Butt discloses wherein the setting circuitry is configured to acquire the search target image from another device (column 15, line 42, “A feature vector is an n-dimensional vector of numerical features (numbers) that represent an image of an object that can be processed by computers.” Column 18, line 19, “In the first method of initiating Appearance Search 500, an image of an object of interest is received 502 at the client 420 ”),
wherein the determination circuitry is configured to detect, as the search target candidate, an object included in the captured video that matches one converted image of the converted images by comparing the converted image converted by the conversion circuitry with the object included in the captured video and determine whether or not part of the detected search target candidate is hidden by a peripheral object (abstract, “The method further comprises comparing the signatures of the identified objects with the signature of the object of interest to generate similarity scores for the identified objects, and transmitting an instruction for presenting on a display one or more of the images of the objects based on the similarity scores.”), and
wherein the search assistance device further comprises search circuitry configured to search for the search target object by comparing the comparison video created by the video creation circuitry with the converted image converted by the conversion circuitry (column 17, line 46, “From either the first method or the second method, a search 506 is then made of the database 414 for candidate feature vectors that have a similarity score, as compared with the feature vector of the object of interest, beyond a threshold, which for example could be 70%. The images of the candidate feature vectors are received 508 and then presented at the client 420 for the user to select 510 the images of the candidate features vectors which are or may be of the object of interest. ”).
However, Mizusawa discloses wherein the conversion circuitry is configured to convert the search target image into the converted images (abstract, “An image synthesis section references the mapping table storage section and converts an image input from an image pickup section into a virtual image viewed from a predetermined virtual eye point.”).
One of ordinary skill in the art would easily know that often objects are obstructed making analysis difficult. However, the object may not be occluded from all viewpoints. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Mizusawa in order to generate views of an object that are not occluded, allowing for more simple and accurate downstream analysis.
Regarding dependent claim 7, the rejection of claim 4 is incorporated herein. Additionally, Butt discloses wherein the setting circuitry includes target image acquisition circuitry configured to acquire, as the search target image, an object image designated based on an operation of a searcher from among object images included in the captured video (column 17, line 43, “ the user searches 514 the
database 414 for an image of the object of interest and retrieves 516 the feature vector of the object of interest which was previously generated when the video was processed for storage in the database 414.”),
wherein the determination circuitry is configured to detect, as the search target candidate, an object included in the captured video that matches one converted image of the converted images by comparing the converted image converted by the conversion circuitry with the object included in the captured video and determine whether or not part of the detected search target candidate is hidden by a peripheral object (column 18, line 40, “A search 606 is then made of the database 414, forward in time from the search time, for candidate feature vectors that have a similarity score, as compared with the feature vector of the object of interest, beyond a threshold, which for example could be 80%. The images of the candidate feature vectors are received 608 and then presented at the client 420 for the user to select 610 one image from the images of the candidate feature vectors which is or may be of the object of interest.” Column 13, line 49, “In a further embodiment, the size of the extracted image is larger than the actual boundaries of the object that has been detected, herein called a Padded cropped bounding box (also referred to as a “Padded Chip”). The Padded cropped bounding box, for example, may be twice the area of the bounding box so that it includes, in whole or in part, objects close to, or overlapping, with the detected foreground visual object.”),
wherein the video creation circuitry is configured to transmit the comparison video to a searcher's terminal device in a case where part of the search target candidate is hidden by the peripheral object (abstract, “transmitting an instruction for presenting on a display one or more of the images of the objects based on the similarity scores.”), and
wherein the search assistance device further comprises communication circuitry configured to transmit the converted image that matches the captured video to the searcher's terminal device in a case where part of the search target candidate is not hidden by the peripheral object (column 17, line 50, “The images of the candidate feature vectors are received 508 and then presented at the client 420 for the user to select 510 the images of the candidate features vectors which are or may be of the object of interest. ”).
However, Mizusawa discloses wherein the conversion circuitry is configured to convert the object image acquired by the target image acquisition circuitry into a plurality of the converted images (abstract, “An image synthesis section references the mapping table storage section and converts an image input from an image pickup section into a virtual image viewed from a predetermined virtual eye point.”),
One of ordinary skill in the art would easily know that often objects are obstructed making analysis difficult. However, the object may not be occluded from all viewpoints. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Mizusawa in order to generate views of an object that are not occluded, allowing for more simple and accurate downstream analysis.
Allowable Subject Matter
Claim 6 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims (and correcting objections to the intervening claims).
Claim 6:
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach methods of performing video search data for target objects which may be occluded.
However, none of them alone or in any combination teaches generating a composite image composed of an image showing a peripheral object hiding an object of interest with a converted image which is the images viewed at multiple different angles, then performing the search for the target by comparing the object from the video with the composite image.
The closest prior art being previously cited Mizusawa discloses at the abstract, “An image synthesis section references the mapping table storage section and converts an image input from an image pickup section into a virtual image viewed from a predetermined virtual eye point.” Mizusawa does disclose generating a converted image which represents an object at different viewing angles.
However, Mizusawa fails to disclose generating a composite image composed of an image showing a peripheral object hiding an object of interest with a converted image which is the images viewed at multiple different angles then performing the search for the target by comparing the object from the video with the composite image.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Publication No. 2013/0176442 to Shuster et al. discloses at paragraph 0018, “a search can be performed for images similar to primary image 210 in which target object 214 is unobstructed by obstruction object 212 (e.g., via an image search for target object 214). In this example, reference image 220 is found, which includes reference object 224, wherein reference object 224 is unobstructed view by obstruction object 212.”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Windsor whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COURTNEY JOAN NELSON/Primary Examiner, Art Unit 2661
Prosecution Insights